CN106934425A - A kind of industrial products counting method based on deep learning - Google Patents

Abstract

The invention discloses a kind of industrial products counting method based on deep learning, belong to machine vision and technical field of video image processing.The present invention is in order to detect the product being blocked, first using the feature that fragmentation training study product is local, then picture is put into network and sets score threshold, score is retained more than the sub- candidate frame of score threshold, the barycenter of all sub- candidate frames is finally carried out into floor projection and upright projection and carries out clustering the substantially barycenter for obtaining product, sub- candidate frame merge the candidate frame of the product for obtaining complete by the Euclidean distance of the barycenter and product barycenter that calculate sub- candidate frame.The present invention is not influenceed by environmental change, it is not necessary to picture is pre-processed, with accuracy of detection very high.

Description

A kind of industrial products counting method based on deep learning

Technical field

The present invention relates to machine vision and technical field of video image processing, depth is based on more specifically to one kind The industrial products counting method of study.

Background technology

Industrial products are possible to that the defects such as oil bottle neglected loading, product missing occur during packing and binding, therefore have The necessary result to casing is detected.Manual detection needs one Quality Inspector of every production line, round-the-clockly on streamline The vanning of process is checked that easy visual fatigue, quality cannot effectively ensure, and needs the huge manpower thing of input Power.

Traditional machine vision technique based on characteristics of image can shoot the picture in vanning from streamline top, then Contour feature based on image judge whether defective, and this alleviates the workload of people to a certain extent.Yet with Site environment is complicated and changeable, the shelter, the change of product color, product in the change of photoenvironment, product shoot it is imperfect, Product is connected etc. all can greatly influence the accuracy rate of this method, cause excessive failing to judge and judging by accident, it is still desirable to manually enter Row secondary detection.

Continued to develop as deep learning is theoretical, its application in actual industrial production is also more and more reliable.Slave Industrial products check problem from the point of view of device study, can be considered a target detection problems.In deep learning field, Ross It is proposed that deep learning network based on Caffe frameworks is used for target detection, the network is drastically increased Girshick etc. The robustness of algorithm of target detection, all has stronger adaptability to illumination and color change, and Detection accuracy obtains very big Raising, the method also has that parameter is few, update convenient advantage, runs into news re -training, and replacing training pattern is Can.Recently also there is related scholar to its further improvement to be applied to different scenes.But for there is the in the case of of blocking, by It is imperfect in target, corresponding feature missing, the network easy missing inspection, traditional method relatively low by score after fl transmission Also cannot detect.

Through retrieval, Chinese Patent Application No. 201610024953.0, the applying date is on January 13rd, 2016, innovation and creation name Referred to as：Method for tracking target based on local feature learning；This application case resolves into target object and background substantial amounts of to be had The local unit of yardstick and Shape invariance, as the training sample of target and background disaggregated model, using deep learning Mode, the local expression of target object and background is gone out from training sample learning.Each given zone in image is judged again Domain belongs to the confidence level of target object, realizes being accurately positioned for target object.The local expression that the study of reason great amount of samples draws Target identification ability with height, with adaptability higher situations such as the method is to target deformation, target occlusion.But should Application case algorithm is more complicated, and general applicability is not strong, can not well be applied to packaging products in boxes detection on streamline.

The content of the invention

1. the invention technical problem to be solved

The problem to be solved in the present invention is：It is existing that method that industrial products check is carried out easily by live ring by machine vision The influence of border change, easily caused inspection and missing inspection, and accuracy of detection is very low.And the target detection network for being based on deep learning can not Detect and be blocked and shoot incomplete product, there is provided a kind of industrial products counting method based on deep learning, the present invention Innovatively propose that the method trained using fragmentation obtains the local feature of target, mesh is then obtained by cluster in detection-phase Target full candidate frame, is not influenceed, it is not necessary to which picture is pre-processed by environmental change, with detection essence very high Degree.

2. technical scheme

To reach above-mentioned purpose, the technical scheme that the present invention is provided is：

A kind of industrial products counting method based on deep learning of the invention, including fragmentation training, Product checking and Sub- candidate frame merges three steps：The product of mark is divided into the sub- candidate frame of some identicals by the fragmentation training stage； One score threshold of Product checking phase sets, score is more than the sub- candidate that the candidate frame of the threshold value is considered as product Frame；In sub- candidate frame fusing stage, by sub- candidate frame barycenter floor projection and upright projection and clustering the matter for obtaining product The heart, the Euclidean distance of barycenter and product barycenter according to sub- candidate frame merges all of sub- candidate frame, obtains the number of product.

Further, a kind of industrial products counting method based on deep learning of the invention, its step is：

Step one, fragmentation training：Product in training set picture is manually marked, a product correspondence on picture One candidate frame, the sub- candidate frame of several identicals is divided into by a candidate frame, every individual sub- candidate frame all representative products One feature of part, the training pictures feeding network that will have been marked is trained；

Step 2, Product checking：Fl transmission will be carried out in product picture input network to be checked, identify current figure All sub- candidate frame as in, and the score of each candidate frame is obtained, remain larger than the sub- candidate frame of set score threshold；

Step 3, the fusion of sub- candidate frame：The barycenter of the sub- candidate frame retained all step 2 carries out floor projection and hangs down Shadow is delivered directly, the projection result according to barycenter in x-axis and y-axis is clustered, obtain the substantially coordinate of product barycenter；Again by meter The Euclidean distance of operator candidate frame and product barycenter, determines the product that sub- candidate frame belongs to.

Further, the process of the fragmentation training sub- candidate frame of acquisition is in step one：When training set is prepared, by original Carry out the artificial candidate frame for marking and be divided into 4 sub- candidate frames, sub- candidate frame generation method is：

In above formula, each element of matrix represents the coordinate in the new candidate frame upper left corner and lower right corner for obtaining, i.e. (x₁, y₁) it is former candidate frame top left co-ordinate, (x₂,y₂) it is former candidate frame bottom right angular coordinate, it is four obtained from big candidate frame in matrix The upper left corner of individual sub- candidate frame and bottom right angular coordinate.

Further, the classification results in step 2 according to softmax graders return to the score of sub- candidate frame.

Further, score threshold as 0.8 is set in step 2.

Further, the barycenter of sub- candidate frame is carried out into floor projection and vertical throwing when step 3 neutron candidate frame is merged Shadow, the projection result according to barycenter in x-axis and y-axis is clustered, and two x-axis coordinates and two y-axis coordinates are obtained after cluster, Four center-of-mass coordinates of product are obtained by permutation and combination, by calculating the Euclidean distance of sub- candidate frame barycenter and product barycenter, Determine which product candidate frame belongs to.

Further, when calculating cluster centre using Euclidean distance in step 3, the object function of algorithm end condition To minimize object to the quadratic sum of the distance of its cluster barycenter：

In formula, K is the number of point to be clustered, C_iIt is point set to be clustered；

By calculating the Euclidean distance of sub- candidate frame barycenter and product barycenter, obtain which product every sub- candidate frame belongs to Product；Minimum Eustachian distance computational methods are：

3. beneficial effect

The technical scheme provided using the present invention, compared with existing known technology, with following remarkable result：

A kind of industrial products counting method based on deep learning of the invention, the basis of sub- candidate frame is obtained in detection On, the characteristics of for the sub- candidate frame of identical product is belonged to close to the barycenter of product, using floor projection and upright projection and gather The method of class is merged to sub- candidate frame, and overcoming network can not detect the defect of the product that is blocked, and realize network application Checked to industrial products.Have benefited from the training method of fragmentation, product can be detected in the form of part in detection, By merging the product so as to obtain complete, Detection accuracy is up to 100%.Its innovation is essentially consisted in：1) for general networking Training complete product can not detect the shortcoming of imperfect product, propose a kind of training method of fragmentation so that incomplete product Product can be detected, and not influenceed by environmental change, it is not necessary to which picture is pre-processed, with detection essence very high Degree.2) propose the method for merging sub- candidate frame to check the product number of vanning, the method using cluster determines product substantially Barycenter.

Brief description of the drawings

Fig. 1 is that the present invention carries out the FB(flow block) that industrial products are checked based on deep learning；

(a)~(d) in Fig. 2 is product image to be checked in the embodiment of the present invention 1；

(a)~(d) in Fig. 3 is the neutron candidate frame testing result figure of the embodiment of the present invention 1；

(a) and (b) in Fig. 4 is the sub- candidate frame barycenter cluster principle figure of the embodiment of the present invention 1；

(a)~(d) in Fig. 5 checks result figure for the industrial products of the embodiment of the present invention 1.

Specific embodiment

To further appreciate that present disclosure, the present invention is described in detail in conjunction with the accompanying drawings and embodiments.

Embodiment 1

With reference to Fig. 1, a kind of industrial products counting method based on deep learning of the present embodiment is trained using fragmentation Network, including fragmentation training, Product checking and sub- candidate frame merge three steps.The fragmentation training stage is by the product of label It is divided into the sub- candidate frame of some identicals；In one score threshold of Product checking phase sets, score is more than the score threshold Candidate frame be considered as product a sub- candidate frame；In sub- candidate frame fusing stage, by sub- candidate frame barycenter level Projection and upright projection and cluster obtain the barycenter of product, and the Euclidean distance of barycenter and product barycenter according to sub- candidate frame is melted All of sub- candidate frame is closed, the number of product is obtained.

The specific implementation process of the present embodiment is as follows：

1st, the fragmentation training stage

Need to prepare training set, the candidate frame of product on picture specially to be trained and picture before network training.One As training set include be target full candidate frame.So training the network for coming can not detect incomplete product. The present embodiment proposes the training method of the fragmentation based on Faster-RCNN, referring to Fig. 2, the present embodiment when training set is prepared, Product in training set picture is manually marked, product one candidate frame of correspondence on picture.And by product Candidate frame is divided into 4 sub- candidate frames of formed objects, and every sub- candidate frame remains a part of feature of product.Sub- candidate Frame generation method is：

In above formula, each element of matrix represents the coordinate in the new candidate frame upper left corner and lower right corner for obtaining, i.e. (x₁, y₁) it is former candidate frame top left co-ordinate, (x₂,y₂) it is former candidate frame bottom right angular coordinate, it is four obtained from big candidate frame in matrix Individual sub- candidate frame.

Each candidate frame can generate 4 sub- candidate frames in the pictures of the present embodiment one, and whole pictures can obtain 16 Sample to be trained so that the sample size of training set is original 4 times.It is incomplete that detection is not only able in this way Product, and target at least one the sub- candidate frame for being not easy to distinguish for light reason can be detected, so Weaker for light this algorithm of scene can also be detected, so as to improve accuracy of detection.

2nd, the Product checking stage

The training pictures feeding network that will have been marked is trained, then the product picture to be detected feeding training that will be photographed Fl transmission is carried out in good network, the classification results according to softmax graders return to the score of sub- candidate frame；According to specific Usage scenario set score threshold (the present embodiment sets score threshold as 0.8), by score more than score threshold sub- candidate Frame retains.Preliminary testing result is obtained, each product is made up of some sub- candidate frames, and phase mutual respect is had between sub- candidate frame It is folded.

3rd, sub- candidate frame fusing stage

The sub- candidate frame that detection is obtained is the candidate frame of product various pieces, it is necessary to sub- candidate frame merge to have obtained Whole product.The present embodiment carries out floor projection and upright projection by by the barycenter of sub- candidate frame (referring to Fig. 4).According to barycenter Projection result in x-axis and y-axis is clustered, and two x-axis coordinates and two y-axis coordinates is obtained after cluster, by arrangement group Conjunction obtains four center-of-mass coordinates of product.By calculating the Euclidean distance of sub- candidate frame and product barycenter, sub- candidate frame category is determined In product.Detailed process is：

When calculating cluster centre using Euclidean distance, the object function of algorithm end condition is minimum object to its cluster matter The quadratic sum of the distance of the heart：

In formula, K is the number of point to be clustered, and Ci is point set to be clustered.

By calculating the Euclidean distance of sub- candidate frame barycenter and product barycenter, can obtain which every sub- candidate frame belong to Product.Minimum Eustachian distance computational methods are：

Product number to be detected can be obtained by the way that sub- candidate frame is merged.

Fig. 2,3,4,5 are the present embodiment implementation result figure, and wherein score threshold is 0.8.(a), (b), (c) in Fig. 3, D () is the sub- candidate frame for detecting, (a) and (b) in Fig. 4 is the schematic diagram of sub- candidate frame barycenter projection and cluster, and Fig. 5 is Testing result figure after sub- candidate frame fusion.It can be seen that the fragmentation training energy that this present embodiment method is used It is enough to detect incomplete product well and do not influenceed by environmental disturbances.

Schematical above that the present invention and embodiments thereof are described, the description does not have restricted, institute in accompanying drawing What is shown is also one of embodiments of the present invention, is actually not limited thereto.So, if one of ordinary skill in the art Enlightened by it, it is similar to the technical scheme without designing for creativeness in the case where the invention objective is not departed from Mode and embodiment, all should belong to protection scope of the present invention.

Claims (7)

1. a kind of industrial products counting method based on deep learning, it is characterised in that：Including fragmentation training, Product checking and Sub- candidate frame merges three steps：The product of mark is divided into the sub- candidate frame of some identicals by the fragmentation training stage； One score threshold of Product checking phase sets, score is more than the sub- candidate that the candidate frame of the threshold value is considered as product Frame；In sub- candidate frame fusing stage, by sub- candidate frame barycenter floor projection and upright projection and clustering the matter for obtaining product The heart, the Euclidean distance of barycenter and product barycenter according to sub- candidate frame merges all of sub- candidate frame, obtains the number of product.

2. a kind of industrial products counting method based on deep learning according to claim 1, its step is：

Step one, fragmentation training：Product in training set picture is manually marked, a product correspondence one on picture Candidate frame, the sub- candidate frame of several identicals, every one of sub- candidate frame all representative products are divided into by a candidate frame Partial feature, the training pictures feeding network that will have been marked is trained；

Step 2, Product checking：Fl transmission will be carried out in product picture input network to be checked, in identifying present image All sub- candidate frame, and obtain the score of each candidate frame, remain larger than the sub- candidate frame of set score threshold；

Step 3, the fusion of sub- candidate frame：The barycenter of the sub- candidate frame retained all step 2 carries out floor projection and vertical throwing Shadow, the projection result according to barycenter in x-axis and y-axis is clustered, and obtains the substantially coordinate of product barycenter；Again by calculating son The Euclidean distance of candidate frame and product barycenter, determines the product that sub- candidate frame belongs to.

3. a kind of industrial products counting method based on deep learning according to claim 2, it is characterised in that：Step one The process that the training of middle fragmentation obtains sub- candidate frame is：When training set is prepared, by the candidate frame average mark of original artificial mark Into 4 sub- candidate frames, sub- candidate frame generation method is：

$(x_1,y_1),(x_2,y_2)\⇒\left[\begin{array}{cc}(x_1,y_1),(\frac{x_1+x_2}{2},\frac{y_1+y_2}{2})& (\frac{x_1+x_2}{2},y_1),(x_2,\frac{y_1+y_2}{2})\\ (x_1,\frac{y_1+y_2}{2}),(\frac{x_1+x_2}{2},y_2)& (\frac{x_1+x_2}{2},\frac{y_1+y_2}{2}),(x_2,y_2)\end{array}\right]$

In above formula, each element of matrix represents the coordinate in the new candidate frame upper left corner and lower right corner for obtaining, i.e. (x₁,y₁) be Former candidate frame top left co-ordinate, (x₂,y₂) it is former candidate frame bottom right angular coordinate, it is four sons obtained from big candidate frame in matrix The upper left corner of candidate frame and bottom right angular coordinate.

4. a kind of industrial products counting method based on deep learning according to Claims 2 or 3, it is characterised in that：Step Classification results in rapid two according to softmax graders return to the score of sub- candidate frame.

5. a kind of industrial products counting method based on deep learning according to claim 4, it is characterised in that：Step 2 It is middle to set score threshold as 0.8.

6. a kind of industrial products counting method based on deep learning according to Claims 2 or 3, it is characterised in that：Step The barycenter of sub- candidate frame is carried out floor projection and upright projection by rapid three neutrons candidate frame when merging, according to barycenter in x-axis and y-axis On projection result clustered, two x-axis coordinates and two y-axis coordinates are obtained after cluster, obtain four by permutation and combination The center-of-mass coordinate of product, by calculating the Euclidean distance of sub- candidate frame barycenter and product barycenter, determines which sub- candidate frame belongs to Product.

7. a kind of industrial products counting method based on deep learning according to claim 6, it is characterised in that：Step 3 When middle use Euclidean distance calculates cluster centre, the object function of algorithm end condition be minimize object to its cluster barycenter away from From quadratic sum：

$min\underset{i=1}{\overset{K}{\Σ}}\underset{x\∈C_i}{\Σ}dist{(c_i,x)}^2$

In formula, K is the number of point to be clustered, C_iIt is point set to be clustered；

By calculating the Euclidean distance of sub- candidate frame barycenter and product barycenter, obtain which product every sub- candidate frame belongs to；Most Small Euclidean distance computational methods are：

$L=min\sqrt{{(x-x_i)}^2+{(y-y_i)}^2},i=1,2,3,4.$