CN110378900A

CN110378900A - The detection method of product defects, apparatus and system

Info

Publication number: CN110378900A
Application number: CN201910711208.7A
Authority: CN
Inventors: 付兴银; 李广
Original assignee: Beijing Maigewei Technology Co Ltd
Current assignee: Beijing Megvii Technology Co Ltd; Beijing Maigewei Technology Co Ltd
Priority date: 2019-08-01
Filing date: 2019-08-01
Publication date: 2019-10-25
Anticipated expiration: 2039-08-01
Also published as: CN110378900B; WO2021017725A1

Abstract

The present invention provides a kind of detection methods of product defects, apparatus and system, are related to detection technique field, this method comprises: obtaining multiple product images of product to be detected；The acquisition angles and/or acquisition position of different product image are different；Defects detection is carried out to product image by the neural network model that training obtains in advance, obtains the defect information of every product image；Defect information includes defective locations and/or defect kind；Initial imperfection testing result is determined based on the defect information of every product image；Initial imperfection testing result is verified；Final defects detection result is determined based on the initial imperfection testing result after verifying.The present invention can effectively promote the accuracy rate of product defects detection.

Description

The detection method of product defects, apparatus and system

Technical field

The present invention relates to detection technique fields, detection method, apparatus and system more particularly, to a kind of product defects.

Background technique

In order to guarantee the quality of finished product, needs to carry out defects detection in process of producing product, be commonly used at present Defects detection mode is the designated position photographs product image in product line, judges that product is based on computer vision No existing defects.But since there may be the bats that partial occlusion or product defects position are not in camera for product surface Situations such as taking the photograph in range leads to the problem of defect leak detection often occur, and since product surface is reflective, also it is easy The problem of existing defect error detection, the comprehensive accuracy rate for causing product defects to detect are lower.

Summary of the invention

In view of this, the purpose of the present invention is to provide a kind of detection method of product defects, apparatus and system, Neng Gouyou Effect promotes the accuracy rate of product defects detection.

To achieve the goals above, technical solution used in the embodiment of the present invention is as follows:

In a first aspect, the embodiment of the invention provides a kind of detection methods of product defects, comprising: obtain product to be detected Multiple product images；The acquisition angles and/or acquisition position of the different product images are different；It is obtained by training in advance Neural network model carries out defects detection to the product image, obtains the defect information of every product image；It is described to lack Sunken information includes defective locations and/or defect kind；Determine that initial imperfection is examined based on the defect information of product image described in every Survey result；The initial imperfection testing result is verified；Final lack is determined based on the initial imperfection testing result after verifying Fall into testing result.

Further, the embodiment of the invention provides the first possible embodiments of first aspect, wherein the acquisition The step of multiple product images of product to be detected, comprising: obtain image capture device under multiple designated positions to it is described to Testing product carries out multiple product images that multi-angled shooting obtains.

Further, the embodiment of the invention provides second of possible embodiments of first aspect, wherein the method Further include: obtain training set of images；Described image training set includes the training image that multiple are marked with defect information；Described in multiple Training image is that image capture device is obtained based on multiple acquisition angles under multiple acquisition positions and each acquisition position 's；Described image training set is input in neural network model to be trained and is trained, the neural network mould after being trained Type.

Further, the embodiment of the invention provides the third possible embodiments of first aspect, wherein the defect Information includes defective locations；The defect information based on product image described in every determines the step of initial imperfection testing result Suddenly, comprising: obtain the mapping relations between every product image and the 3D model of the product to be detected pre-established； Based on the mapping relations, the defective locations that every product image detects are projected on the 3D model, based on throwing Penetrate the defective locations that position determines the 3D model；Initial imperfection testing result is determined according to the defective locations of the 3D model.

Further, the embodiment of the invention provides the 4th kind of possible embodiments of first aspect, wherein the product Image is 2D image；Between acquisition every product image and the 3D model of the product to be detected pre-established The step of mapping relations, comprising: choose a target 2D image from 2D image described in multiple, obtain the target 2D image pair The depth image answered；Establish the mapping relations between the target 2D image and the depth image；Based on point cloud registration algorithm Point cloud registering is carried out to the 3D model of the depth image and the product to be detected pre-established, obtains the depth image With the mapping relations between the 3D model；Based on the mapping relations between the target 2D image and the depth image, with And the mapping relations between the depth image and the 3D model, it obtains between the target 2D image and the 3D model Mapping relations；Acquisition position based on the target 2D image and the mapping between the acquisition position of other 2D images are closed System and the mapping relations between the target 2D image and the 3D model, obtain other 2D images and the 3D mould Mapping relations between type.

Further, the embodiment of the invention provides the 5th kind of possible embodiments of first aspect, wherein described to be based on Launching position determines the step of defective locations of the 3D model, comprising: projects the defective locations of every product image It is clustered to the obtained launching position of 3D model, the defective locations of the 3D model is determined based on cluster result.

Further, the embodiment of the invention provides the 6th kind of possible embodiments of first aspect, wherein it is described will be every The defective locations of Zhang Suoshu product image are projected to the obtained launching position of 3D model and are clustered, and are based on cluster result The step of determining the defective locations of the 3D model, comprising: it is described to determine that the defective locations of every product image are projected to The obtained launching position of 3D model；Multiple target launching positions are determined based on the obtained launching position and predetermined manner, The cluster centre of the central point of multiple target launching positions is determined as to the central point of the defective locations of the 3D model；Its In, the coordinate of the cluster centre is the average value of the coordinate of the central point of multiple target launching positions；The default side Formula includes: to regard all launching positions as target launching position；Alternatively, the launching position in preset range is carried out Merge, obtains target launching position；Alternatively, there will be the launching position of lap to merge, target launching position is obtained.

Further, the embodiment of the invention provides the 7th kind of possible embodiments of first aspect, wherein it is described will be every The defective locations of Zhang Suoshu product image are projected to the step of obtained launching position of 3D model is clustered, comprising: Determine that the defective locations of every product image are projected to the central point of the obtained launching position of 3D model；Repetition is held The preset clustering algorithm of row, until the central point of multiple launching positions has sorted out and cluster centre no longer changes；It is described Clustering algorithm are as follows: the central point of specified quantity is randomly selected from the central point of multiple launching positions as cluster centre； The point of proximity of current each cluster centre is searched from the central point of multiple launching positions using KD-Tree algorithm, The point of proximity of current each cluster centre is classified as one kind, and has the cluster centre of coincidence to merge point of proximity； For the central point that do not sort out, new cluster centre is chosen；Wherein, the central point of the point of proximity and the launching position it Between distance be less than preset threshold.

Further, the embodiment of the invention provides the 8th kind of possible embodiments of first aspect, wherein described to institute State the step of initial imperfection testing result is verified, comprising: for each defective locations of the 3D model, according to product figure The production of defect corresponding with the defective locations of the 3D model should occur in picture and the mapping relations between the 3D model, determination The theoretical quantity of product image；Based on the defect information of product image described in every, determination is detected through the neural network model The product image with the corresponding defect of the defective locations actual quantity；Calculate the actual quantity and the theoretical quantity The first ratio；Include if first ratio lower than default first numerical value, determines in the initial imperfection testing result The corresponding defect of the defective locations is false defect.

Further, the embodiment of the invention provides the 9th kind of possible embodiments of first aspect, wherein described to institute State the step of initial imperfection testing result is verified, comprising: according between product image described in every and the 3D model Mapping relations are projected to every product image for the defective locations of the 3D model are inverse；To occur and the 3D model The product image of the corresponding inverse projection defect of defective locations is determined as defect image；Calculate the quantity of the defect image and through institute State the second ratio of the quantity for the product image with the corresponding defect of the defective locations that neural network model detects；If Second ratio is lower than default second value, determines that the defective locations for including in the initial imperfection testing result are corresponding Defect is false defect.

Further, the embodiment of the invention provides the tenth kind of possible embodiments of first aspect, wherein described to be based on The step of initial imperfection testing result after verifying determines final defects detection result, comprising: based on the initial imperfection after verifying Testing result rejects false defect included in the initial imperfection testing result；It will be described initial scarce after rejecting false defect Testing result is fallen into as final defects detection result.

Second aspect, the embodiment of the present invention also provide a kind of detection device of product defects, comprising: image collection module, For obtaining multiple product images of product to be detected；The acquisition angles and/or acquisition position of the different product images are different； Defects detection module is obtained for carrying out defects detection to the product image by the neural network model that training obtains in advance To the defect information of product image described in every；The defect information includes in defective locations, flaw size and defect kind It is one or more；Initial results determining module, for determining that initial imperfection is examined based on the defect information of product image described in every Survey result；Authentication module, for being verified to the initial imperfection testing result；Final result determining module, for being based on Initial imperfection testing result after verifying determines final defects detection result.

The third aspect, the embodiment of the invention provides a kind of detection system of product defects, the system comprises: image is adopted Acquisition means, processor and storage device；Described image acquisition device, for acquiring product image；It is stored on the storage device There is computer program, the computer program executes such as the described in any item sides of first aspect when being run by the processor Method.

Fourth aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage Computer program is stored on medium, the computer program is executed when being run by processor described in above-mentioned any one of first aspect Method the step of.

The embodiment of the invention provides a kind of detection methods of product defects, apparatus and system, can obtain multiple acquisitions Angle and/or the different product image of acquisition position, and defects detection is carried out to multiple product images by neural network model, The defect information of every product image is obtained, the defect information for being then based on every product image determines initial imperfection detection knot Fruit；Initial imperfection testing result is further verified later, and final lack is determined based on the initial imperfection testing result after verifying Fall into testing result.On the one hand aforesaid way provided in this embodiment is based on multiple acquisition angles and/or acquisition position got It sets different product image and carries out defects detection, namely defects detection carried out to product from different position angles, therefore can be compared with The probability of product defects leak detection is reduced well, on the other hand defects detection result can be verified, therefore can be preferable Ground reduces the probability of product defects error detection, therefore the comprehensive accuracy rate for improving defects detection result.

Other feature and advantage of the embodiment of the present invention will illustrate in the following description, alternatively, Partial Feature and excellent Point can deduce from specification or unambiguously determine, or the above-mentioned technology by implementing the embodiment of the present invention can obtain Know.

To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate Appended attached drawing, is described in detail below.

Detailed description of the invention

It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art Embodiment or attached drawing needed to be used in the description of the prior art be briefly described, it should be apparent that, it is described below Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor It puts, is also possible to obtain other drawings based on these drawings.

Fig. 1 shows the structural schematic diagram of a kind of electronic equipment provided by the embodiment of the present invention；

Fig. 2 shows a kind of detection method flow charts of product defects provided by the embodiment of the present invention；

Fig. 3 shows a kind of product image acquisition schematic diagram to be detected provided by the embodiment of the present invention；

Fig. 4 shows a kind of defect information verification method flow chart of product image provided by the embodiment of the present invention；

Fig. 5 shows a kind of defective locations projection schematic diagram provided by the embodiment of the present invention；

Fig. 6 shows a kind of verifying schematic diagram of initial imperfection testing result provided by the embodiment of the present invention；

Fig. 7 shows a kind of inverse projection schematic diagram of defective locations provided by the embodiment of the present invention；

Fig. 8 shows the defect inspection method flow chart of another kind product provided by the embodiment of the present invention；

Fig. 9 shows the 3D illustraton of model of the product to be detected after a kind of projection of defect provided by the embodiment of the present invention；

Figure 10 shows a kind of structural block diagram of the detection device of product defects provided by the embodiment of the present invention.

Specific embodiment

In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention Technical solution be clearly and completely described, it is clear that described embodiments are some of the embodiments of the present invention, rather than Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise Under every other embodiment obtained, shall fall within the protection scope of the present invention.

In view of the accuracy rate of existing product defects detection technique is lower, in order to effectively promote the standard of product defects detection True rate can be applied to need to product the embodiment of the invention provides a kind of detection method of product defects, apparatus and system The occasion that defect is detected, applied to occasions such as industrial productions.It describes in detail below to the embodiment of the present invention.

Embodiment one:

Firstly, referring to Fig.1 come describe the detection methods of product defects for realizing the embodiment of the present invention a kind of, device and The exemplary electronic device 100 of system.

The structural schematic diagram of a kind of electronic equipment as shown in Figure 1, electronic equipment 100 include one or more processors 102, one or more storage devices 104, input unit 106, output device 108 and image collecting device 110, these components It is interconnected by bindiny mechanism's (not shown) of bus system 112 and/or other forms.It should be noted that electronic equipment shown in FIG. 1 100 component and structure be it is illustrative, and not restrictive, as needed, the electronic equipment also can have other Component and structure.

The processor 102 can use digital signal processor (DSP), field programmable gate array (FPGA), can compile At least one of journey logic array (PLA) example, in hardware realizes that the processor 102 can be central processing unit (CPU) or one or more of the processing unit of other forms with data-handling capacity and/or instruction execution capability Combination, and can control other components in the electronic equipment 100 to execute desired function.

The storage device 104 may include one or more computer program products, and the computer program product can To include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easy The property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non- Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage medium On can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter institute The client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meter Can also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/or The various data etc. generated.

The input unit 106 can be the device that user is used to input instruction, and may include keyboard, mouse, wheat One or more of gram wind and touch screen etc..

The output device 108 can export various information (for example, image or sound) to external (for example, user), and It and may include one or more of display, loudspeaker etc..

Described image acquisition device 110 can be shot the desired image of user (such as photo, video etc.), such as, to be checked Survey the product image of product；And captured image is stored in the storage device 104 for the use of other components.

Illustratively, show for realizing the detection methods of product defects according to an embodiment of the present invention, apparatus and system Example electronic equipment may be implemented as the intelligent terminals such as computer, defect detection equipment.

Embodiment two:

A kind of detection method flow chart of product defects shown in Figure 2, S202~step that this approach includes the following steps Rapid S206:

Step S202 obtains multiple product images of product to be detected；The acquisition angles and/or acquisition of different product image Position is different.

Specifically, multiple above-mentioned product images can be respectively from multiple acquisition positions around product to be detected, with Multiple product images that identical acquisition angles are got；Either from the identical acquisition position around product to be detected, Multiple product images got respectively with different acquisition angles；It can also be from multiple acquisition positions around product to be detected It sets, multiple product images got respectively with different acquisition angles.In practical applications, above-mentioned acquisition position can root It is determined according to the shape and structure of product to be detected and the tilt angle of product surface to be detected.Preferably, set acquisition Position can cover the outer surface of product to be detected, so as to comprehensively get the appearance details photo of product to be detected. Above-mentioned acquisition angles can be according to the distance between image capture device and product to be detected size and product to be detected itself Size setting acquisition angles range, allow image capture device collected within the scope of the acquisition angles of setting to Detail pictures of the testing product under each acquisition angles.

Step S204 carries out defects detection to product image by the neural network model that training obtains in advance, obtains every Open the defect information of product image；Defect information includes defective locations and/or defect kind.

In practical applications, it can obtain can recognize that the minds of product defects to be detected in a manner of off-line training to use in advance Through network model, the defects of multiple product images of above-mentioned extraction are detected using trained neural network model, and are identified All defect information in every product image out, it only includes defective locations which, which can be, and only includes defect Type can also be including defective locations and defect kind.Since multiple above-mentioned product images are around product to be detected The image of multiple angle acquisitions of multiple acquisition positions, than the appearance images for more fully having collected product to be detected, then benefit Defects detection is carried out with trained neural network model, by the more powerful data-handling capacity of neural network, Ke Yiyou Effect prevents the defect missing inspection problem of product surface to be detected.

Drawbacks described above position mentioned in the present embodiment can embody specific location of the defect on product and lack The information such as sunken shape and size, drawbacks described above type can be such as various types of scuffing, bubble, spot dark/bright, spot etc. Defect.

Step S206 determines initial imperfection testing result based on the defect information of every product image.

Lacking in every product image of product to be detected can be identified using above-mentioned trained neural network model Information is fallen into, trained neural network model can also mark the defects of every product image position and defect kind, from And the defects detection of every product image is obtained as a result, since multiple above-mentioned product images are that product to be detected is outer than more comprehensively Image is seen, the defects detection result based on each product image can obtain the defect letter than more comprehensive product surface to be detected Breath, to determine that initial imperfection testing result, initial imperfection testing result may include scarce based on more comprehensive defect information Fall into all defect for the product to be detected that information is showed.

Above-mentioned initial imperfection testing result, which can be, is presented all defect information detected using tabular form；It can also be with It is to be presented in the form of including the image set for all product images for detecting defect；It can also be using product to be detected The all defect detected is presented in 3D model, such as, can establish 3D model (or the 3D point cloud mould of product to be detected Type), by the defects of every product image position by the projection of flat image to 3D model, obtain lacking in each product image Sunken position projects the defective locations on 3D model, to pass through the defect of 3D model display product to be detected.

Step S208 verifies initial imperfection testing result.

In view of product surface is likely to occur phenomena such as reflective, such as it is easy for hot spot mistake to be considered that defect is (namely pseudo- Defect), it is happened so as to cause erroneous detection.Therefore the present embodiment can further verify the true and false of initial imperfection testing result, with Screen out false defect.

Step S210 determines final defects detection result based on the initial imperfection testing result after verifying.

It can be the detection of false defect by the verification result of initial imperfection testing result in a kind of specific embodiment As a result it rejects, the initial imperfection testing result after rejecting false defect is final defects detection as a result, in this way, can Effectively improve erroneous detection problem existing in the prior art.

The detection method of the said goods defect provided in an embodiment of the present invention can obtain multiple acquisition angles and/or adopt Collect the different product image in position, and defects detection is carried out to multiple product images by neural network model, obtains every production The defect information of product image, the defect information for being then based on every product image determine initial imperfection testing result；Later to first Beginning defects detection result is further verified, and determines final defects detection result based on the initial imperfection testing result after verifying. On the one hand aforesaid way provided in this embodiment is the production different based on multiple acquisition angles got and/or acquisition position Product image carries out defects detection, namely carries out defects detection to product from different position angles, therefore can preferably reduce production On the other hand the probability of product defect leak detection can verify defects detection result, therefore can preferably reduce product The probability of defect error detection, therefore the comprehensive accuracy rate for improving defects detection result.

In order to enable multiple product images for getting product to be detected that product surface feature, this reality is more fully presented Example is applied when obtaining multiple product images of product to be detected, available image capture device is treated under multiple designated positions Testing product carries out multiple product images that multi-angled shooting obtains namely product image is image capture device in multiple positions What multiple angles were shot.

For ease of understanding, following present a kind of specific embodiment of multiple product images for obtaining product to be detected, It specifically can refer to step (1)~(3) execution:

(1) it obtains image capture device and treats the video that testing product progress rotary taking obtains under multiple designated positions Stream.

In practical applications, multiple designated positions can be set around product to be detected, passed through on designated position The image capture devices such as camera, camera acquire the video flowing of product to be detected.Preferably, above-mentioned multiple designated positions can To cover the surface of product to be detected, make each surface of product to be detected can face image capture device.Image Acquisition Equipment can obtain video flowing of the product to be detected under different shooting angles in specified location rotary taking, in practical application In, schematic diagram is acquired referring to product image to be detected shown in Fig. 3, " cross " the polygon A in figure is product to be detected, to be checked Symbolic three designated positions c1, c2 and c3 illustrated for image capture device to be arranged above product are surveyed, such as Fig. 3 institute Show, image capture device above-mentioned designated position c1, c2 and c3 acquire product image when, can with the angle of rotary taking image, To be continuously taken image of the product to be detected under different acquisition angle, to form video flowing, the angle of rotation can root It is determined according to the tilt angle of product surface to be detected.In specific implementation, it can be and carried out only with an image capture device Shooting, the camera site of the image capture device is converted based on above three designated position, makes the image capture device at three Different designated positions carries out Image Acquisition, is also possible to be shot using three image capture devices, each Image Acquisition Equipment is placed in a designated position and carries out Image Acquisition respectively.It is understood that Fig. 3 is only a kind of exemplary illustration, letter Three designated positions are shown, but are not limited only to three designated positions in practical applications, can also be detected according to actual defects Demand is arranged multiple designated positions in the surrounding of product to be detected and places image capture device.

(2) video frame of preset quantity is extracted respectively from the corresponding video flowing in each designated position.

It is understood that in the product video flowing to be detected of acquisition include omnidirectional shooting multiple image, due to Video frame included in video flowing is large number of, therefore the present embodiment is distinguished from the video flowing that each specified location is shot The video frame for extracting preset quantity not only can reflect out the image that different angle takes, but also can promote subsequent defect Detection efficiency, the video frame extracted may include the image of the product all angles to be detected taken in specified location, Correspondingly, the video flowing of each designated position shooting requires to extract the image of preset quantity as video frame.For example, image It includes 100 figures that acquisition equipment, which rotates the video flowing shot after 30 degree of angles in each specified location of product to be detected, Picture can extract 10 images from the video flowing that 100 images are formed and form video frame, above-mentioned 10 extracted figure 3 degree of 10 images are divided between shooting angle, as can be more comprehensively to obtain product to be detected under all angles Image can extract 10 images if image capture device shoots video flowing in 3 specified locations from each video flowing As video frame, therefore 30 images are extracted altogether.

(3) using the video frame of extraction as multiple product images of product to be detected.Due to the above-mentioned video frame being drawn into For image of the product to be detected under different specified camera sites and different shooting angles, it more can comprehensively pass through above-mentioned view Frequency frame observes appearance information of the product to be detected under different angle, therefore, can be using the video frame being drawn into as to be checked Survey multiple product images of product.

It in another embodiment specific implementation mode, can be in multiple predeterminated positions and each predeterminated position is with multiple default Angle acquisition obtains product image.Image capture device can also be to be detected with preset rotation angle acquisition at designated position The image of product is carried out for example, image capture device can be in each predeterminated position with rotation 30 degree of acquisitions, one photo Shooting namely each predeterminated position are provided with 12 predetermined angles, to form multiple the product figures for acquiring product to be detected Picture.

In order to more accurately identify the defects of the said goods image, the mind after obtaining training is present embodiments provided Specific embodiment through network model, reference can be made to following steps 1 and step 2:

Step 1: obtaining training set of images；Training set of images includes that multiple are marked with defect information and (alternatively referred to as carry Defective labels) training image；Multiple training images are that image capture device is based on multiple acquisition positions and each acquisition position Under multiple acquisition angles obtain.It, can be using as complete as possible in order to improve the accuracy rate of neural network model identification defect The training set of images in face trains neural network model, and training image included in training set of images can be using Image Acquisition Equipment treats testing product and carries out what comprehensive full angle shooting, collecting arrived, and training image is embodied with product to be detected and owns Defective locations, defect kind for being likely to occur etc. promote mind to enable neural network model comprehensively to carry out defect study Defects detection accuracy rate through network model.

Step 2: training set of images is input in neural network model to be trained and is trained, the nerve after being trained Network model.The training set of images for being marked with all defect information is inputted to neural network model to be trained, by such as anti- Neural network model is trained to propagation algorithm, the neural network model after being trained, the neural network mould after training Type can effectively identify the defect of product surface to be detected according to the product image of input.

In view of in actual defect inspection process, it is also possible to defect erroneous detection problem occur, such as, due to the mirror of product Face reflection etc. influences, and the white light of reflection may be mistakenly considered product defects.Therefore, it is lacked in order to further enhance product testing Sunken accuracy rate, the detection method of the said goods defect provided in this embodiment further include the defect verification step to product.Ginseng According to a kind of defect information verification method flow chart of product image shown in Fig. 4, wherein drawbacks described above information includes defective bit Set, for product defect verifying the following steps are included:

Step S402, the mapping obtained between every product image and the 3D model of the product to be detected pre-established are closed System.

Specifically, the said goods image be 2D image, image capture device acquisition product to be detected 2D image with to It is that there are certain mapping relations, and image capture device is collected in different designated positions between the 3D model of testing product 2D image and the mapping relations of the 3D model of product to be detected be different from, need first to get the 2D image of product to be detected Then mapping relations between the 3D model of product to be detected carry out subsequent defect mapping again.

Following present reflecting between a kind of 3D model of product to be detected for obtaining every product image and pre-establishing The specific embodiment of relationship is penetrated, reference can be made to following (1)~(5) are realized:

(1) a target 2D image is chosen from multiple 2D images, obtains the corresponding depth image of target 2D image.Its In, target 2D image, which can be, to be extracted from multiple 2D images that the initial designated position of product to be detected is shot, It is also possible to extract from multiple 2D images that any other designated position is shot, above-mentioned depth image can be use RGB-D camera (for example, it may be kinect camera) shoots while obtaining target 2D image and depth image, is also possible to distinguish Shoot product to be detected simultaneously using RGB camera and depth camera, collect respectively product to be detected target 2D image and Corresponding depth image.

(2) mapping relations between target 2D image and depth image are established.According to the internal reference of image capture device camera Matrix can establish the mapping relations of target 2D image and depth image.Wherein, the internal reference matrix of camera can pass through camera mark It is fixed to obtain, calibration is carried out to camera it is, for example, possible to use Zhang Zhengyou calibration algorithm and obtains internal reference matrix.By the internal reference matrix of camera Know the angle of z-axis and x, y-axis, therefore by the z-axis coordinate in depth image you can learn that corresponding x, y are sat in target 2D image Mark.

(3) cloud is carried out based on 3D model of the point cloud registration algorithm to depth image and the product to be detected pre-established to match Standard obtains the mapping relations between depth image and 3D model.The 3D model of product to be detected be it is known (can be by preparatory The CAD model for the product to be detected established carries out Model Reconstruction and obtains the 3D point cloud model of product to be detected, abbreviation 3D model), The 3D point cloud coordinate of product namely to be detected is known.Point cloud chart picture can be obtained by above-mentioned depth image, utilize PPF (Point Pair Feature) point cloud registration algorithm or other point cloud registration algorithms are to above-mentioned point cloud chart picture and 3D point cloud model Point cloud registering is carried out, to establish the mapping between the 3D model of product to be detected and the corresponding depth image of target 2D image Relationship.

By taking PPF algorithm as an example, this gives a kind of specific embodiment parties that point cloud registering is carried out using PPF algorithm Formula: needing to initially set up 3D point cloud model midpoint to the Hash table of PPF Feature Descriptor, and the PPF for then calculating point cloud chart picture is special Sign description, and corresponding point is found in Hash table, by corresponding point to PPF feature calculation candidate's pose, then will wait The initial pose transformation of bit selecting appearance cluster calculation.It also needs after obtaining initial pose transformation using iteration closest approach algorithm (Iterative Closest Point, abbreviation ICP) is refined to pose and (is calculated accurate pose), finally realizes depth map As with being registrated between 3D point cloud model.

(4) based between mapping relations and depth image between target 2D image and depth image and 3D model Mapping relations obtain the mapping relations between target 2D image and 3D model (that is, 3D point cloud model).

In the specific implementation, the mapping relations formula between target 2D image and 3D model is P_m=T_mcP_c, wherein P_mFor The coordinate of point in 3D point cloud model, P_cFor the coordinate of pixel in target 2D image, T_mcTo indicate 3D point cloud model and target 2D The mapping matrix of image mapping relations.It, can will be in the target 2D image of product to be detected according to above-mentioned mapping relations formula Defective locations are mapped to the corresponding position in 3D point cloud model, and obtain defective locations information (such as the seat of the point of Defect Edge Mark, defect center point coordinate, defect extreme coordinates etc.).

(5) acquisition position based on target 2D image and the mapping relations between the acquisition position of other 2D images, and Mapping relations between target 2D image and 3D model obtain the mapping relations between other 2D images and 3D model.In reality In, the acquisition position of target 2D image and the acquisition position of other 2D images can be obtained based on the shifting principle of mechanical arm Between mapping relations, so as to based between target 2D image and 3D model mapping relations and mechanical arm shifting principle obtain The mapping relations between 2D image and 3D model shot to other acquisition positions.

Step S404 is based on mapping relations, the defective locations that every product image detects is projected on 3D model, base The defective locations of 3D model are determined in launching position.

In one embodiment, the defective locations of every product image can be projected to the obtained projection of 3D model Position is clustered, and the defective locations of 3D model are determined based on cluster result.Pass through obtained 2D image in abovementioned steps (4) With the mapping relations between 3D model, the defects of every product image (the 2D image of product i.e. to be detected) position can be thrown Be mapped on 3D model, defective locations referring to Figure 5 project schematic diagram, in figure biggish " cross " the polygon a in top be to The 3D model of testing product, illustrates product image p1, p2 and p3 of 3 products to be detected in Fig. 5, in every product image all Defective locations (illustrating with roundlet) is marked.By the shooting, collecting position of defect presented in every product image and acquisition Angle is different from, therefore the position that defect is presented on the image may be variant, and the identical defect in multiple product images is thrown Also there can be position deviation when being mapped on 3D model, to will not be completely coincident, therefore, it is also desirable to cluster to launching position To determine the defective locations on 3D model.

Present embodiments provide the main cluster mode of following two:

The first cluster mode:

Determine that the defective locations of every product image are projected to the obtained launching position of 3D model first；It is then based on To launching position and predetermined manner determine multiple target launching positions, later by the poly- of the central point of multiple target launching positions Class center is determined as the central point of the defective locations of 3D model；Wherein, the coordinate of cluster centre is multiple target launching positions The average value of the coordinate of central point.

Above-mentioned predetermined manner includes: to regard all launching positions as target launching position；Alternatively, will be at a distance of default model Launching position in enclosing merges, and obtains target launching position；Alternatively, will there is the launching position of lap to merge, Obtain target launching position.That is, the cluster centre of the central point of all launching positions can be determined as to lacking for 3D model Fall into the central point of position, the projection position that launching position adjacent on 3D model can also be merged or there will be lap It sets and merges, the cluster centre of the central point of the launching position after merging is determined as to the center of the defective locations of 3D model Point.

Second of cluster mode:

Determine that the defective locations of every product image are projected to the central point of the obtained launching position of 3D model first；So After repeat preset clustering algorithm, until the central point of multiple launching positions has been sorted out and cluster centre no longer changes. Above-mentioned clustering algorithm are as follows: the central point of specified quantity is randomly selected from the central point of multiple launching positions as cluster centre； The point of proximity of current each cluster centre is searched from the central point of multiple launching positions using KD-Tree algorithm, it will be currently every The point of proximity of a cluster centre is classified as one kind, and has the cluster centre of coincidence to merge point of proximity；For not sorting out Central point, choose new cluster centre；Remove the cluster centre that point of proximity quantity is less than preset quantity；Wherein, point of proximity with The distance between central point of launching position is less than preset threshold.After the final determining cluster centre being no longer changed Using cluster centre as the central point of the defective locations of 3D model.For ease of understanding, it is based on the cluster mode, for details, reference can be made to such as Under step (1)~step (5):

Step (1): assuming that the defects of multiple product images position, which is projected to the defects of 3D model position, n point, N point is established into a K-D tree, can be convenient using K-D tree and look for point of proximity.K-D tree (abbreviation of k-dimensional tree) is It is a kind of divide k dimension data space data structure, be mainly used in hyperspace critical data search (such as: range searching and Nearest neighbor search).

Step (2): m point is randomly selected in n point of drawbacks described above position, the m cluster centre as initialization. Central point (m of specified quantity are randomly selected in the step namely the aforementioned central point (n central point) from multiple launching positions Central point) it is used as cluster centre.

Step (3): it for each cluster centre, is looked in n all points apart from small by the K-D tree in step (1) In the point of proximity of threshold value, point of proximity is classified as a kind of set, if m neighbor point has overlapping, two classes are classified as one kind.The step It is rapid namely aforementioned search closing on for current each cluster centre from the central point of multiple launching positions using KD-Tree algorithm Point, and the point of proximity of each cluster centre is classified as one kind, and there is the cluster centre of coincidence to merge point of proximity.

Step (4): the operation to repeat the above steps in (3), for the point of each cluster centre, using KD-Tree algorithm The point for being less than threshold value from cluster centre is classified as one kind by the point of proximity for looking for cluster centre, is merged class if fruit has overlapping, directly No longer change to all classes.

Step (5): if there are also not classified points in n point, for random choosing again in unclassified central point New cluster centre is taken, the operation for (1)~step (4) that repeat the above steps, until all point is all classified.

In addition, in practical applications, in above-mentioned cluster mode, preset threshold can also be less than in the points for being attributed to a class When, such is removed, to realize faster cluster operation.

Step S406 determines initial imperfection testing result according to the defective locations of 3D model.

Initial imperfection testing result can be the 3D model for presenting the defective locations obtained after above-mentioned cluster, in order to Family intuitively can observe the position of each defect and defect on 3D model by 3D model.

In order to promote the accuracy of verifying initial imperfection testing result, following two is present embodiments provided to initial imperfection The embodiment that testing result is verified:

Verification mode one: referring in particular to following steps (1)~(4):

(1) for each defective locations of 3D model, according to the mapping relations between product image and 3D model, determination is answered There is the theoretical quantity of the product image of defect corresponding with the defective locations of 3D model.

It is possible, firstly, to understand, it is specific on product that the defective locations presented on 3D model can embody defect The information such as the shape and size of position and defect, when such as defect kind is " scuffing ", defective locations may present thin-line-shaped； When defect kind is " bubble ", round may be presented in defective locations.Think clustered on 3D model in the present embodiment to obtain The corresponding defect of each defective locations (certainly, the corresponding defect kind of different defective locations may be identical or different). Assuming that 3D model is clustered to show 3 defective locations, then it is assumed that each defective locations correspond to a defect, on the 3D model Coenosarc reveals 3 defects.

Due to the acquisition position and acquisition angles of each product image be it is known, according between product image and 3D model Mapping relations, each defective locations that can extrapolate can theoretically take on 3D model correspond to the Image Acquisition of defect Position and image capturing angle, thus the production taken under the multiple images acquisition position and multiple images acquisition angles extrapolated The quantity of product image is that the theoretical quantity of the product image of defect corresponding with the defective locations of 3D model (theoretically has How many product images should take the defect).

Initial imperfection testing result shown in Figure 6 verifies schematic diagram, and the 3D model of product to be detected is illustrated in Fig. 6 Solid black lines circle on a, 3D model a is that one of defective locations in initial imperfection testing result (only show one in Fig. 6 A defective locations, but not representing initial imperfection testing result only includes a defective locations), which can also be with Regard the corresponding defect (being assumed to be hot spot) in the black circles position (i.e. defective locations) as.Go out through what theory was calculated Now the product image of defect corresponding with the defective locations of 3D model is p1, p2 and p3 in Fig. 6, the void on p1, p2 and p3 Line circle is to calculate position of the corresponding defect of the defective locations in product image, namely calculate and shoot Theoretical quantity to the product image of hot spot is 3.

(2) defect information based on every product image, determining has the defective bit through what neural network model detected Set the actual quantity of the product image of corresponding defect.

When determining the product image with the corresponding defect of the defective locations detected through neural network model, this reality It applies example and provides the following two kinds implementation:

Mode one: if the defective locations of the defect on 3D model are by clustering after the defective locations mapping in product image It obtains, then it is assumed that there is defect corresponding with defective locations on the 3D model in the product image.

Mode two: it based on the mapping relations between 3D model and product image, is incident upon the defective locations of 3D model are inverse In product image；For a product image, if the inverse position being incident upon in the product image of the defective locations of 3D model It sets and is detected (referred to as against launching position) with neural network model and being overlapped between the defective locations that are labeled in the product image Rate is higher than preset threshold, then it is assumed that the product image has the corresponding defect of defective locations on 3D model.

By the above-mentioned means, more accurately can easily determine have the defective locations on 3D model are corresponding to lack The actual quantity of sunken product image.

As shown in fig. 6, product image p1 and p3 present solid line circle respectively, the solid line circle namely neural network model The corresponding defect of the defective locations of the 3D model detected, namely actually photographed hot spot through what neural network model detected Product image actual quantity be 2.

(3) the first ratio of actual quantity and theoretical quantity is calculated.

As shown in fig. 6, the actual quantity of the product image in Fig. 6 with the corresponding defect of the defective locations is 2, calculate Now the theoretical quantity of the product image of defect corresponding with the defective locations of 3D model is 3, then the first ratio is 2/3.

(4) if the first ratio is lower than default first numerical value, the defective bit for including in initial imperfection testing result is determined Setting corresponding defect is false defect.

If the first ratio that the calculated defective locations correspond to defect is less than default first numerical value (default first numerical value Size can be configured according to the shape or structure of product to be detected), then illustrate that the defective locations on 3D model are corresponding Defect be determined as false defect.

Verification mode two: referring in particular to following steps (1)~(4):

(1) it according to the mapping relations between every product image and 3D model, is projected to the defective locations of 3D model are inverse Every product image.

Based on the mapping relations of the available inverse projection of above-mentioned mapping relations formula, to obtain the defects of 3D model Inverse position is projected to the defects of every product image position.Illustratively, referring to the inverse projection signal of defect as shown in Figure 7 Scheme, biggish " cross " polygon in top is the 3D model of product to be detected in figure, and the solid line circle on 3D model a is cluster The defective locations obtained afterwards, p1, p2 and p3 in figure are the product image of three products to be detected, and the defects of 3D model passes through After inverse projection, the inverse position being incident upon in product image of the defective locations by 3D model presented in multiple product images is obtained It sets, as shown in the dashed circle in Fig. 7 in product image p1, p2 and p3.

(2) product image for inverse projection defect corresponding with the defective locations of 3D model occur is determined as defect image. Inverse solid line circle of the position as shown in the dashed circle in Fig. 7, in Fig. 7 being incident upon in product image of the defective locations of 3D model For through original defective locations in neural network model detects and marks out product image.

When the inverse position (dashed circle) being incident upon in product image of the defective locations of 3D model with it is original in product image The area coincidence factor of defective locations (solid line circle) when being greater than default coincidence factor threshold value, then it is assumed that the production where the dashed circle Product image is the inverse image for projecting defect corresponding with the defective locations of 3D model occur, also as defect image.When 3D model Defective locations inverse be incident upon original defective locations (solid line circle in the position in product image (dashed circle) and product image Circle) be overlapped without area or when coincidence factor is less than default coincidence factor threshold value, then it is assumed that the product image where the dashed circle Not there is the image of inverse projection defect corresponding with the defective locations of 3D model, namely is not belonging to defect image.For example, Fig. 7 In product image p1 and p2 in 3D model the inverse position (dashed circle) being incident upon in product image of defective locations and produce The area coincidence factor of original defective locations (solid line circle) is greater than default coincidence factor threshold value in product image, therefore by product image P1 and p2 are determined as defect image；The inverse position being incident upon in product image of the defective locations of 3D model in product image p3 The area coincidence factor of original defective locations (solid line circle) is less than default coincidence factor threshold value in (dashed circle) and product image, Therefore product image p3 is not the image of inverse projection defect corresponding with the defective locations of 3D model, i.e. product image p3 is not scarce Fall into image.

(3) calculate the quantity of defect image with through neural network model detect with the corresponding defect of the defective locations Product image quantity the second ratio.

By taking Fig. 7 as an example, defect image quantity is 2, has the defective locations corresponding through what neural network model detected The quantity of the product image of defect is 3, then the second ratio is 2/3.It is understood that being detected due to product to be detected There may be false defects in defect, for example, product to be detected is metal product, it is to be detected when image capture device acquires image Certain positions of product are reflective to cause to be determined as existing defects by mistake, due to the reflective position of the product image of multiple angle acquisitions It is variant, when there are the defects of inverse projection defect against causing when projecting in multiple product images for the reflective false defect taken The quantity of image is less than the quantity of multiple above-mentioned product images.

(4) if above-mentioned second ratio is lower than default second value, determine that the defect information of product image is fake information.When When defect image and the quantity ratio (the second ratio) of multiple product images are less than default second value, that is, defect image The corresponding defect information of defect image for being unsatisfactory for the inverse projection defect of the default ratio is then determined as pseudo- letter by negligible amounts Breath.

In order to keep final defects detection result more accurate, the present embodiment can be detected based on the initial imperfection after verifying and be tied Fruit rejects false defect included in initial imperfection testing result；Using reject false defect after initial imperfection testing result as Final defects detection result.In this way, influence of the erroneous detection to final defects detection result is effectively prevented, is further mentioned The accuracy of defects detection is risen.In practical applications, it is based on aforementioned drawback detection method, the present embodiment is briefly provided such as figure The defect inspection method flow chart of product shown in 8 specifically comprises the following steps S802 to step S810:

Step S802, uncalibrated image acquire the mapping relations of equipment and product to be detected.That is, obtaining image capture device Mapping relations between the product image of shooting and the 3D model of product to be detected.

Step S804, it is to be detected with the comprehensive acquisition of multiple shooting angle in multiple acquisition positions using image capture device The product image of product.In practical applications, multiple product images can be uniformly extracted from above-mentioned video flowing, what is extracted is more Can product image be opened can be the image that product external surfaces to be detected are more comprehensively showed from each orientation.

Step S806 detects the defects of multiple product images based on neural network model.The defect includes defective bit Set with defect kind etc..

Step S808 projects the defects of multiple product images on the 3D model of product to be detected, by projection Defect carry out clustering the defective locations on determining 3D model.

Step S810 verifies the corresponding defect of defective locations on 3D model.

For ease of understanding, below in conjunction with the 3D illustraton of model of the product to be detected after defect shown in Fig. 9 projection, one kind is provided Specific application example:

For example, defective locations show three defects on the 3D model of product to be detected, it is (true to lack to be respectively as follows: defect d1 Fall into, defect d1 be in obvious position), defect d2 (false defect, such as the hot spot of reflective generation), d3 (real defect, it is scarce It falls into d3 and is in remote position).

Image capture device takes 100 product images of product to be detected with multiple acquisition positions and acquisition angles, Since the position defect d1 is obvious, it is assumed that wherein there is 50 product images to take defect d1；Since the position defect d3 is remote, so May only specific position and angle can just take defect d3, it is therefore assumed that there is 10 product images to take defect d3； Simultaneously because d2 is not real defect, but hot spot, it can only be that special angle can be just photographed because reflective, it is therefore assumed that its In there are 5 product images to take defect d2.The defect that the said goods image is detected and marked through neural network model The defective locations of d1, defect d2 and defect d3 are projected to 3D model respectively, as shown in figure 9, since defect d1 and defect d3 are true Real defect, so the launching position being incident upon in 3D model can compare concentration；And defect d2 is due to being false defect (such as light Spot), deviation is had in the defective locations that different acquisition angles take, thus the launching position projected in 3D model also can It is more dispersed.The position being incident upon on 3D model defect d1, defect d2 and defect d3 in multiple product images is clustered, The defect d1 showed on 3D model, the defective locations of defect d2 and defect d3 are obtained, and then to the defective locations on 3D model Corresponding defect is verified.

(such as aforementioned authentication mode one) in one embodiment is closed according to the mapping between product image and 3D model System, the theoretical quantity 53,50 and 10 of the product image of defect d1, defect d2 and defect d3 should occur in determination respectively, and through nerve The actual quantity 50,5 and 10 for the product image with defect d1, defect d2 and defect d3 that network model detects, then distinguish The first ratio 50/53 of defect d1, the first ratio 5/50 of defect d2 is calculated, the first ratio 10/10 of defect d3 is preset If the first numerical value is set as 0.8, it is determined that defect d2 of first ratio lower than 0.8 is false defect.

(such as aforementioned authentication mode two) in another embodiment will cluster defect d1, the defect of generation on 3D model The defective locations of d2 and defect d3 are inverse to be projected in product image, and the inverse projection defect of each product image is obtained, and inverse projection lacks The defective locations in product image can be detected with original nerve network model and be labeled in certain coincidence factor by falling into, it is assumed that defect For d1 against when being projected in product image, only 48 images meet preset area Duplication requirement, then defect d1 is corresponding lacks Falling into image is 48.Assuming that defect d2 is against when being projected in product image, only 1 image meets preset area Duplication and wants It asks, then the corresponding defect image of defect d2 is 1.Assuming that defect d3 is against when being projected in product image, only 9 images meet Preset area Duplication requirement, then the corresponding defect image of defect d3 is 9.Because detecting through neural network it is found that product Only have 50 image takings to arrive defect d1 in image, then it is assumed that there are the defect images of d1 (and can be described as visual image) to be 50；There are 5 image takings to arrive defect d2 in product image, then it is assumed that there are the defect image of d2 (visual image) be 5；Product There are 10 image takings to arrive defect d3 in image, then it is assumed that there are the defect image of d3 (visual image) be 10.Therefore, above-mentioned The second ratio of defect d1 is 48/50；The second ratio of defect d2 is 1/5；The second ratio of defect d3 is 9/10.If setting Default second ratio is 0.6, it is determined that defect d2 of second ratio lower than 0.6 is false defect.

The detection method of the said goods defect provided in an embodiment of the present invention carries out product in different location/angle complete Orientation defects detection by projecting the defects of 2D image in 3D model, and is clustered and is obtained on the basis of leak-stopping inspection Defect on 3D model, then project the defect on 3D model is inverse on 2D image, thus the defect that 2D image detection is gone out into One step card, has been effectively relieved the erroneous detection problem in defects detection, the above method can be effectively reduced false dismissal probability, and pick Except erroneous detection, synthesis improves the accuracy and robustness of industrial products defects detection.

Embodiment three:

Corresponding to the detection method of product defects provided in embodiment two, the embodiment of the invention provides a kind of products The detection device of defect, a kind of structural block diagram of the detection device of product defects shown in Figure 10, which includes following Module:

Image collection module 11, for obtaining multiple product images of product to be detected；The acquisition angle of different product image Degree and/or acquisition position are different.

Defects detection module 12, for carrying out defect inspection to product image by the neural network model that training obtains in advance It surveys, obtains the defect information of every product image；Defect information includes one of defective locations, flaw size and defect kind Or it is a variety of.

Initial results determining module 13, for determining that initial imperfection detects based on the defect information of product image described in every As a result.

Authentication module 14, for being verified to initial imperfection testing result.

Final result determining module 15, for determining final defects detection knot based on the initial imperfection testing result after verifying Fruit.

The detection device of the said goods defect provided in an embodiment of the present invention can obtain multiple acquisition angles and/or adopt Collect the different product image in position, and defects detection is carried out to multiple product images by neural network model, obtains every production The defect information of product image, the defect information for being then based on every product image determine initial imperfection testing result；Later to first Beginning defects detection result is further verified, and determines final defects detection result based on the initial imperfection testing result after verifying. On the one hand aforesaid way provided in this embodiment is the production different based on multiple acquisition angles got and/or acquisition position Product image carries out defects detection, namely carries out defects detection to product from different position angles, therefore can preferably reduce production On the other hand the probability of product defect leak detection can verify defects detection result, therefore can preferably reduce product The probability of defect error detection, therefore the comprehensive accuracy rate for improving defects detection result.

In one embodiment, above-mentioned image collection module 11 is further used for obtaining image capture device multiple Testing product is treated under designated position carries out multiple product images that multi-angled shooting obtains.

In one embodiment, above-mentioned apparatus further include: network training module, for obtaining training set of images；Image Training set includes the training image that multiple are marked with defect information；Multiple training images are that image capture device is based on multiple acquisitions What multiple acquisition angles under position and each acquisition position obtained；Training set of images is input to neural network model to be trained In be trained, the neural network model after being trained.

In one embodiment, drawbacks described above information includes defective locations；Initial results determining module 13, is further used Mapping relations between every product image of acquisition and the 3D model of the product to be detected pre-established；Based on mapping relations, The defective locations that every product image detects are projected on 3D model, the defective bit of 3D model is determined based on launching position It sets；Initial imperfection testing result is determined according to the defective locations of 3D model.

In a specific embodiment, product image is 2D image；Above-mentioned initial results determining module 13, is further used In choosing a target 2D image from multiple 2D images, the corresponding depth image of target 2D image is obtained；Establish target 2D figure Mapping relations between picture and depth image；Based on point cloud registration algorithm to depth image and the product to be detected that pre-establishes 3D model carries out point cloud registering, obtains the mapping relations between depth image and 3D model；Based on target 2D image and depth map The mapping relations between mapping relations and depth image and 3D model as between, obtain target 2D image and 3D model it Between mapping relations；Acquisition position based on target 2D image and the mapping relations between the acquisition position of other 2D images, with And the mapping relations between target 2D image and 3D model, obtain the mapping relations between other 2D images and 3D model.

In one embodiment, above-mentioned initial results determining module 13 is further used for lacking every product image Sunken position is projected to the obtained launching position of 3D model and is clustered, and the defective locations of 3D model are determined based on cluster result.

In one embodiment, above-mentioned initial results determining module 13 is further used for determining every product image Defective locations are projected to the obtained launching position of 3D model；Multiple targets are determined based on obtained launching position and predetermined manner The cluster centre of the central point of multiple target launching positions is determined as the central point of the defective locations of 3D model by launching position； Wherein, the coordinate of cluster centre is the average value of the coordinate of the central point of multiple target launching positions；Predetermined manner includes: by institute Some launching positions are used as target launching position；Alternatively, the launching position in preset range is merged, mesh is obtained Mark launching position；Alternatively, there will be the launching position of lap to merge, target launching position is obtained.

In one embodiment, above-mentioned initial results determining module 13 is further used for determining every product image Defective locations are projected to the central point of the obtained launching position of 3D model；Preset clustering algorithm is repeated, until multiple The central point of launching position has been sorted out and cluster centre no longer changes；Clustering algorithm are as follows: from the central point of multiple launching positions In randomly select the central point of specified quantity as cluster centre；Using KD-Tree algorithm from the central point of multiple launching positions The point of proximity of current each cluster centre is classified as one kind, and will closed on by the middle point of proximity for searching current each cluster centre Point has the cluster centre of coincidence to merge；For the central point that do not sort out, new cluster centre is chosen；Wherein, point of proximity The distance between central point of launching position is less than preset threshold.

In one embodiment, above-mentioned authentication module is further used for each defective locations for 3D model, according to The product of defect corresponding with the defective locations of 3D model should occur in mapping relations between product image and 3D model, determination The theoretical quantity of image；Based on the defect information of every product image, determining has the defective bit through what network model detected Set the actual quantity of the product image of corresponding defect；Calculate the first ratio of actual quantity and theoretical quantity；If the first ratio Value determines that the corresponding defect of the defective locations for including in initial imperfection testing result is false defect lower than default first numerical value.

In one embodiment, above-mentioned authentication module is further used for according between every product image and 3D model Mapping relations, be projected to every product image for the defective locations of 3D model are inverse；The defective locations pair with 3D model to occur The product image for the inverse projection defect answered is determined as defect image；The quantity for calculating defect image is detected with through neural network model Second ratio of the quantity of the product image with the corresponding defect of the defective locations out；If the second ratio is lower than default the Two numerical value determine that the corresponding defect of the defective locations for including in initial imperfection testing result is false defect.

In one embodiment, above-mentioned final result determining module 15, it is initial scarce after being further used for based on verifying Testing result is fallen into, false defect included in initial imperfection testing result is rejected；By the initial imperfection detection after rejecting false defect As a result it is used as final defects detection result.

The detection device of the said goods defect provided in an embodiment of the present invention carries out defect to product from different position angles On the basis of detection prevents missing inspection, obtained in 3D model by projecting in 3D model and clustering the defects of 2D image position Defective locations, then the defects of 3D model inverse position is projected into the mode on 2D image, to the defect information detected into A step of advancing card, effectively prevents the erroneous detection problem in defects detection, the above method can be effectively prevented missing inspection, and pick Except erroneous detection, the robustness of industrial products defects detection is improved.

The technical effect of device provided by the present embodiment, realization principle and generation is identical with previous embodiment, for letter It describes, Installation practice part does not refer to place, can refer to corresponding contents in preceding method embodiment.

Example IV:

Corresponding to method and apparatus provided by previous embodiment, the embodiment of the invention also provides a kind of product defects Detection system, the system include: image collecting device, processor and storage device.

Above-mentioned image collecting device, for acquiring product image.

Computer program is stored on above-mentioned storage device, the computer program executes such as when being run by processor 92 Method provided by preceding method embodiment.

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description Specific work process, can be with reference to the corresponding process in previous embodiment, and details are not described herein.

The computer program product of the detection methods of product defects provided by the embodiment of the present invention, apparatus and system, packet The computer readable storage medium for storing program code is included, the instruction that said program code includes can be used for executing previous methods Method as described in the examples, specific implementation can be found in embodiment of the method, and details are not described herein.

In addition, in the description of the embodiment of the present invention unless specifically defined or limited otherwise, term " installation ", " phase Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can To be mechanical connection, it is also possible to be electrically connected；It can be directly connected, can also can be indirectly connected through an intermediary Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition Concrete meaning in invention.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention. And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.

Finally, it should be noted that embodiment described above, only a specific embodiment of the invention, to illustrate the present invention Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair It is bright to be described in detail, those skilled in the art should understand that: anyone skilled in the art In the technical scope disclosed by the present invention, it can still modify to technical solution documented by previous embodiment or can be light It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover in protection of the invention Within the scope of.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.

Claims

1. a kind of detection method of product defects characterized by comprising

Obtain multiple product images of product to be detected；The acquisition angles and/or acquisition position of the different product images are different；

Defects detection is carried out to the product image by the neural network model that training obtains in advance, obtains every product The defect information of image；The defect information includes defective locations and/or defect kind；

Initial imperfection testing result is determined based on the defect information of product image described in every；

The initial imperfection testing result is verified；

Final defects detection result is determined based on the initial imperfection testing result after verifying.

2. the method according to claim 1, wherein the step of multiple product images for obtaining product to be detected Suddenly, comprising:

It obtains image capture device and multiple that multi-angled shooting obtains is carried out to the product to be detected under multiple designated positions Product image.

3. the method according to claim 1, wherein the method also includes:

Obtain training set of images；Described image training set includes the training image that multiple are marked with defect information；Multiple described instructions Practicing image is that image capture device is obtained based on multiple acquisition angles under multiple acquisition positions and each acquisition position；

Described image training set is input in neural network model to be trained and is trained, the neural network mould after being trained Type.

4. method according to any one of claims 1 to 3, which is characterized in that the defect information includes defective locations；Institute State the step of defect information based on product image described in every determines initial imperfection testing result, comprising:

The mapping relations that every product image is obtained between the 3D model of the product to be detected that pre-establishes；

Based on the mapping relations, the defective locations that every product image detects are projected on the 3D model, base The defective locations of the 3D model are determined in launching position；

Initial imperfection testing result is determined according to the defective locations of the 3D model.

5. according to the method described in claim 4, it is characterized in that, the product image is 2D image；Every institute of the acquisition The step of mapping relations for product image being stated between the 3D model of the product to be detected that pre-establishes, comprising:

A target 2D image is chosen from 2D image described in multiple, obtains the corresponding depth image of the target 2D image；

Establish the mapping relations between the target 2D image and the depth image；

A cloud is carried out based on 3D model of the point cloud registration algorithm to the depth image and the product to be detected pre-established Registration, obtains the mapping relations between the depth image and the 3D model；

Based on the mapping relations and the depth image and the 3D mould between the target 2D image and the depth image Mapping relations between type obtain the mapping relations between the target 2D image and the 3D model；

Acquisition position based on the target 2D image and the mapping relations between the acquisition position of other 2D images, and Mapping relations between the target 2D image and the 3D model obtain between other 2D images and the 3D model Mapping relations.

6. according to the method described in claim 4, it is characterized in that, the defect for determining the 3D model based on launching position The step of position, comprising:

The defective locations of every product image are projected to the obtained launching position of 3D model to cluster, are based on Cluster result determines the defective locations of the 3D model.

7. according to the method described in claim 6, it is characterized in that, the defective locations by every product image project It is clustered to the obtained launching position of 3D model, the step of the defective locations of the 3D model is determined based on cluster result Suddenly, comprising:

Determine that the defective locations of every product image are projected to the obtained launching position of 3D model；

Multiple target launching positions are determined based on the obtained launching position and predetermined manner, and multiple targets are projected into position The cluster centre for the central point set is determined as the central point of the defective locations of the 3D model；Wherein, the seat of the cluster centre Mark is the average value of the coordinate of the central point of multiple target launching positions；

The predetermined manner includes: to regard all launching positions as target launching position；Alternatively, will be in preset range Launching position merge, obtain target launching position；Alternatively, there will be the launching position of lap to merge, obtain Target launching position.

8. according to the method described in claim 6, it is characterized in that, the defective locations by every product image project The step of being clustered to the obtained launching position of 3D model, comprising:

Determine that the defective locations of every product image are projected to the central point of the obtained launching position of 3D model；

Repeat preset clustering algorithm, until multiple launching positions central point sorted out and cluster centre no longer Change；

The clustering algorithm are as follows: the central point of specified quantity is randomly selected from the central point of multiple launching positions as poly- Class center；Current each cluster centre is searched from the central point of multiple launching positions using KD-Tree algorithm The point of proximity of current each cluster centre is classified as one kind by point of proximity, and by point of proximity have the cluster centre of coincidence into Row merges；For the central point that do not sort out, new cluster centre is chosen；Wherein, the point of proximity and the launching position The distance between central point is less than preset threshold.

9. according to the method described in claim 4, it is characterized in that, described verify the initial imperfection testing result Step, comprising:

Each defective locations of the 3D model are determined according to the mapping relations between product image and the 3D model The theoretical quantity of the product image of defect corresponding with the defective locations of the 3D model should occur；

Based on the defect information of product image described in every, determining has the defective bit through what the neural network model detected Set the actual quantity of the product image of corresponding defect；

Calculate the first ratio of the actual quantity and the theoretical quantity；

If first ratio is lower than default first numerical value, the defective bit for including in the initial imperfection testing result is determined Setting corresponding defect is false defect.

10. according to the method described in claim 4, it is characterized in that, described verify the initial imperfection testing result The step of, comprising:

For each defective locations of the 3D model, closed according to the mapping between product image described in every and the 3D model System is projected to every product image for the defective locations of the 3D model are inverse；

The product image for inverse projection defect corresponding with the defective locations of the 3D model occur is determined as defect image；

The quantity for calculating the defect image has the defective locations are corresponding to lack with what is detected through the neural network model Second ratio of the quantity of sunken product image；

If second ratio is lower than default second value, the defective bit for including in the initial imperfection testing result is determined Setting corresponding defect is false defect.

11. according to the method described in claim 4, it is characterized in that, described true based on the initial imperfection testing result after verifying The step of fixed final defects detection result, comprising:

Based on the initial imperfection testing result after verifying, false defect included in the initial imperfection testing result is rejected；

Using the initial imperfection testing result after rejecting false defect as final defects detection result.

12. a kind of detection device of product defects characterized by comprising

Image collection module, for obtaining multiple product images of product to be detected；The acquisition angles of the different product images And/or acquisition position is different；

Defects detection module, for carrying out defect inspection to the product image by the neural network model that training obtains in advance It surveys, obtains the defect information of every product image；The defect information includes defective locations and/or defect kind；

Initial results determining module, for determining initial imperfection testing result based on the defect information of product image described in every；

Authentication module, for being verified to the initial imperfection testing result；

Final result determining module, for determining final defects detection result based on the initial imperfection testing result after verifying.

13. a kind of detection system of product defects, which is characterized in that the system comprises: it image collecting device, processor and deposits Storage device；

Described image acquisition device, for acquiring product image；

Computer program is stored on the storage device, the computer program is executed when being run by the processor as weighed Benefit requires 1 to 11 described in any item methods.

14. a kind of computer readable storage medium, computer program, feature are stored on the computer readable storage medium The step of being, the described in any item methods of the claims 1 to 11 executed when the computer program is run by processor.