CN104268538A - Online visual inspection method for dot matrix sprayed code characters of beverage cans - Google Patents

Abstract

The invention provides an online visual inspection method for sprayed code characters at the bottoms of beverage cans, wherein the method is based on machine vision. The online visual inspection method comprises the step of conducting initial positioning on character regions by adopting an MSER method and the step of conducting fine positioning according to a connected region method. According to the characteristics of the dot matrix sprayed code characters, segmentation is conducted by the adoption of a dot matrix sprayed code character segmentation algorithm, and segmentation accuracy is guaranteed. During character recognition, recognition is conducted according to a convolution neural network recognition method, detection precision is improved while real-time performance is guaranteed, and the requirements for the high real-time performance and high accuracy of detection of the dot matrix sprayed code characters at the bottoms of the beverage cans can be completely met.

Description

The online visible detection method of a kind of pop can dot matrix coding character

Technical field

The present invention relates to and utilize machine vision to pop can dot matrix coding character on-line checkingi field, specifically refer to the image processing method of coding character recognition at the bottom of a kind of that be applied to industry spot, that high real-time requires pop can tank.

Background technology

Along with the sane raising of living standards of the people, China's food beverage industry maintains the impetus increased fast, with pop can be the demand of container also in continuous increase, the product quality in process of manufacture is the embodiment that brand names are worth, and is also directly connected to the vital interests of consumer simultaneously.In order to the raising of product quality, need to detect product information, follow the trail of.Therefore, how in real time pop can dot matrix coding character recognition to be detected, so that rejecting substandard product is in time the problem needing solution badly.

Current pop can coding character online measuring technique research is mainly divided into three parts:

Part I is the location of character zone.At present, industrial character recognition usually under off-line state first frame select ROI region, during on-line checkingi setting ROI region in carry out detections identification.Effectively can reduce the interference of ground unrest in this way, locating effect is better.But on actual industrial production line, pop can easily rotates in coding process, cannot arrange fixing ROI region, how orienting pop can character is accurately the key identified.

Part II is Character segmentation technology.Continuous whole coding character adopts traditional shadow casting technique to split usually, and dot character can produce a lot of breakpoint in projection process, adopts traditional projection localization technology accurately to split.Therefore before segmentation, the demi-inflation that intercharacter ruptures couples together more by the plavini that adopts, so that later stage segmentation, because Character segmentation directly has influence on the effect of character recognition, therefore Character segmentation technology is the emphasis of research always.

Part III is Feature extraction and recognition technology, at present, character recognition mostly is extracts many architectural features, statistical nature in advance, carry out correlation analysis again, these features are better for traditional print character recognition effect, but for the dot matrix coding character of easy fracture, easily distortion, recognition effect is not good.Convolutional neural networks (Convolutional Neural Network, be called for short CNN) for the degree of depth study a kind of mode of learning, machine is utilized to go to learn high-quality coding feature, avoid feature extraction complicated in tional identification algorithm and data reconstruction processes, and there is good fault-tolerant ability and parallel processing capability.

Summary of the invention

The object of the invention is to provide a kind of for dot matrix coding character real-time detection method at the bottom of pop can tank, can meet the requirement of industry spot high real-time, high discrimination.

For reaching this object, technical scheme of the present invention is as follows: identifying is divided into off-line training process and on-line checkingi process, and off-line training process mainly comprises character pre-processing separating character at the bottom of pop can tank, and builds sorter with the character after segmentation; On-line checkingi process mainly comprises the segmentation of character at the bottom of pop can tank, then is input to by the character of segmentation in the sorter that builds and identifies.Specifically comprise the following steps:

Off-line training step:

(1) obtain character picture at the bottom of pop can tank, adjust the mechanical parameters such as camera lens aperture, focal length, ball integration light source is installed, pictures taken above at the bottom of pop can tank, obtains pop can tank base map picture to be detected.

(2) character zone location, first MSER method (most stable extremal region) is adopted to process image, coarse positioning is carried out to character, morphological dilations method is adopted to carry out expansion process again, area-method is adopted carefully to locate, and by minimum enclosed rectangle determination character zone and sense of rotation.

(3) Character segmentation, the character picture rotation correction will tilted by affined transformation and linear interpolation, is adopted dot character segmentation module that character zone is divided into single character, and is normalized to 28*28 size.

(4) sorter training, by the single character split, unification is sorted out, and sets up coding character repertoire, and adopts convolutional neural networks learning method to train the character repertoire established, and obtains CNN sorter.

The on-line checkingi stage:

(1) character picture at the bottom of Real-time Obtaining pop can tank, adjusts the mechanical parameters such as camera lens aperture, focal length, installs ball integration light source, pictures taken, Real-time Obtaining pop can tank to be detected base map picture above at the bottom of pop can tank.

(2) character locating and segmentation, MSER operation is carried out to the image obtained, obtain character picture, adopt morphological approach expansion character, character zone is oriented according to character zone area information, adopt Minimum Enclosing Rectangle method by character zone rotation correction again, and adopt dot character segmentation module to split to postrotational image.

(3) character normalization split is 28*28 size by character recognition, then sends in the CNN sorter that trained, carries out feedforward study, thus judge character attibute, output character information in the sorter built.

(4) character information and production line administrative system are carried out data transmission, realize the management of date of manufacture to pop can and lot number.Perform testing result by electric control system, to do not conform in management system or lack print, the pop can of None-identified of biting rejects.

Beneficial effect of the present invention: the invention provides character machining algorithm at the bottom of a kind of pop can tank.For the feature of dot character at the bottom of pop can tank, MSER and morphology area-method is adopted to position character zone, the interference of delete character yardstick, rotational transform.Devise dot character partitioning algorithm for dot character simultaneously, ensure that the accuracy of dot character in segmentation.In character recognition, the sorter adopting convolutional neural networks to build identifies.This kind of character identifying method can ensure the high efficiency of character locating at the bottom of industry spot pop can tank, segmentation and identification, meets in detection algorithm real-time, the requirement that accuracy is high.

Accompanying drawing explanation

Fig. 1 pop can character zone of the present invention positioning flow figure

Fig. 2 dot character segmentation of the present invention process flow diagram

Fig. 3 convolutional neural networks training classifier of the present invention process flow diagram

Fig. 4 on-line detecting system overall flow figure of the present invention

Embodiment

For making the object, technical solutions and advantages of the present invention clearly understand, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in further detail.

The present invention is coding character detection method at the bottom of a kind of pop can tank, testing process is divided into off-line training process and on-line checkingi process, under off-line state, character zone location process flow diagram is as Fig. 1, Character segmentation process flow diagram is as Fig. 2, character after segmentation adopts convolutional neural networks training classifier, training process flow diagram is as Fig. 3, first character zone is oriented during on-line checkingi, again character zone is divided into single character, and the character split is input in sorter, output character information, on-line checkingi process flow diagram is as Fig. 4.

Further, off-line procedure specific implementation step is:

Step one, image sequence obtain

According to the requirement of character recognition accuracy of detection, adjust the mechanical parameters such as camera lens aperture, focal length, obtain character picture at the bottom of pop can tank.

Step 2, character zone are located

(1) MSER algorithm process is adopted to the image obtained, uses different gray thresholds to obtain character zone to image binaryzation:

(1.1) adopt BinSort algorithm to given image, by the gray-scale value sequence in image, time complexity is 0 (n);

(1.2) extremal region merges and chooses, and according to descending or ascending order rule, the gray-value pixel after sequence is put into image, adopts Component Tree method to obtain each extremal region.

(1.3) on the Component Tree formed, from certain node, branch is searched for, according to the relative change rate in formulae discovery region in tonal range, in the branch of this Component Tree, try to achieve local minimum, obtain stable MSER region, be the character zone of coarse positioning.

(2) adopt 3*3 rectangular configuration to perform morphological dilation to image, and each connected component labeling in image is added up the area of connected domain.Arrange character zone attribute, the connected domain of area between (s1, s2) is character zone, and concrete numerical value adjusts according to different model pop can.

(3) rotate character zone, calculated the anglec of rotation of character zone by minimum enclosed rectangle, and adopt affined transformation and linear interpolation that the character zone of inclination is corrected to horizontal direction.

Step 3, Character segmentation

Character at the bottom of pop can tank is generally dot matrix coding character, there is gap between character, and adopt dot character segmentation module to split, segmentation module is mainly divided into following components:

(1) character row segmentation

(1.1) first adopt sciagraphy to the capable projection of character zone, add up its projection value Prover [i].

$\mathrm{Prover}\left[i\right]=\underset{j=1}{\overset{\mathrm{row}}{\mathrm{\Σ}}}I(i,j)$

Wherein row is the line number of image.

(1.2) pop can character is dot character, often usually there is gap between row, the basis of projection localization adopts bellow expansion method, when adding up the projection value at every bit place, if by this point centered by, containing impact point in the scope being bound threshold value with bellow expansion number of times, then the projection value of these row is added 1, after selecting bellow expansion, the i-th projective representation being listed in horizontal direction is:

$\mathrm{ProVer}\left[j\right]=\underset{i=1}{\overset{\mathrm{row}}{\mathrm{\Σ}}}\mathrm{DilateGray}(i,j)$

$\mathrm{DilateGray}(i,j)=\left\{\begin{array}{ccc}1& \mathrm{if}& \underset{m=j-\mathrm{SEC}}{\overset{j+\mathrm{SEC}}{\mathrm{\Σ}}}I(i,m)\≠0\\ 0& \mathrm{if}& \underset{m=j-\mathrm{SEC}}{\overset{j+\mathrm{SEC}}{\mathrm{\Σ}}}I(i,m)=0\end{array}\right.$

Wherein I (i, j) represents the pixel value at bianry image (i, j) place, and SEC represents bellow expansion number of times.

(1.3) segmentation threshold is set.According to the image after bellow expansion, segmentation threshold threshold_row is set, when projection value is less than threshold_row after bellow expansion, is row split position.

(2) character row segmentation

(2.1) adopt vertical projection method, the overall width W in significant character region can be counted, according to the character number n of priori, calculate the mean breadth x of character.

(2.2) the bellow expansion number of times of row character is set, carries out bellow expansion.Segmentation threshold threshold_col is set, carries out coarse segmentation.Often row dot character interval is less, has partial character to be sticked together after bellow expansion, adopts conglutination segmentation method to split:

1. 2 Characters Stucks: all subregion character duration be partitioned into and average character duration x are compared, group peak width is greater than 1.3 times of average character duration, when being less than 2.2 times, is judged as two Connection operators.Again from initial split position, find out the minimum value of waveform projection value between distance split position 0.8x to 1.2x, the column position corresponding to it is split position.

2. 3 Characters Stucks: all subregion character duration be partitioned into and average character duration x are compared, group peak width is greater than 2.3 times of average character duration, when being less than 3.2 times, is judged as three Connection operators.From initial split position, find out between distance split position 0.8x to 1.2x and between 1.8x to 2.2x, the minimum value of waveform projection value, two column positions corresponding to it are the position of twice segmentation.

Step 3, structure sorter

Convolutional neural networks training algorithm is mainly divided into three phases: build network structure, propagated forward stage and back-propagation phase.

Step1, structure network structure

The convolutional neural networks that the present invention builds adopts 7 Rotating fields, comprise input layer, two convolutional layers, two down-sampling layers, full articulamentum and output layers, the initial stage of training first builds network structure, adopt different little random numbers to carry out initialization to the weights of network simultaneously, general in [-1 ~ 1] scope, biased initialization is set to 0.

Step2, propagated forward stage

(1) input layer input target image Y, and the target vector d of correspondence.Be convolutional layer as illustrated C1, C3 layer in 2, convolutional layer carries out two-dimensional convolution by convolution kernel to input picture, and adds upper offset, then is obtained by nonlinear activation function:

$Y_j^n=f(\underset{i=1}{\overset{S}{\mathrm{\Σ}}}{Y}_i^{n-1}*W_{\mathrm{ij}}^n+{\mathrm{\φ}}_j^n)$

Wherein: n represents the number of plies, S represents the unit number of n layer, W _ijthe convolution (weights) of the 5*5 size of connection i-th input picture and a jth output image, φ _jbe the threshold value (being biased) of output image j, f (*) is RELU function:

$f\left(x\right)=\max (x,0)=\max (\underset{i=1}{\overset{S}{\mathrm{\Σ}}}{Y}_i^{n-1}*W_{\mathrm{ij}}^n+{\mathrm{\φ}}_j^n,0)$

(2) in the present invention, S2, S4 are down-sampling layer, and the mode of down-sampling adopts stochastic pooling to sample, and formula is:

$Y_t={\mathrm{\Σ}}_{j\∈R_t}{p}_j{Y}_j$

Wherein: r _tfor the window size of layer of sampling, be generally 2*2 size, Y _jfor the element value of sample window.

(3) in the present invention, M5 layer is full connection layer by layer, after full attended operation is carried out to M5 layer, and the actual output O of computational grid layer F6 _k:

$O_k=f(\underset{t=1}{\overset{l}{\mathrm{\Σ}}}{V}_{\mathrm{tk}}{Y}_t+{\mathrm{\θ}}_k)$

Wherein k is output layer unit number, θ _kfor the threshold value (being biased) of output unit, l is the unit number of M5, V _jkfor connecting the convolution of full articulamentum and output layer, wherein f (*) is softmax function.

Step3, back-propagation phase

Back-propagation phase adopts gradient descent method oppositely to adjust the weights and threshold of each layer, and statistics total error function is: when E≤ε (arranging least error parameter), training terminates, and is preserved by weights and threshold.At this moment network structure parameters is stablized, and sorter is formed.

Further, in line process specific implementation step be:

Step one, image sequence obtain

According to the requirement of character recognition real-time online accuracy of detection, adjust the mechanical parameters such as camera lens aperture, focal length, character picture at the bottom of Real-time Obtaining pop can tank.

Step 2, pretreatment operation

First MSER process is carried out to the image obtained, obtain the character picture of coarse positioning.Adopt morphological dilations method to expand, and each connected component labeling in image is added up the area of connected domain.According to the area attribute arranged under off-line state, orienting character zone, is horizontal direction by Minimum Enclosing Rectangle method by the character zone rotation correction of inclination.And adopt dot character segmentation module that character zone is divided into single character.

Step 3, character recognition

The character normalization split is 28*28 size by character recognition, then sends in the CNN sorter that trained, carries out feedforward study, thus judge character attibute, output character information in the sorter built.

Step 4, faulty materials are rejected

The character information of output and production line administrative system are carried out data transmission, realizes the management of date of manufacture for pop can and lot number.Perform testing result by electric control system, for do not conform in management system or lack print, the defective pop can of None-identified of biting rejects.

Claims (4)

1. one kind based on coding character online test method at the bottom of the pop can tank of machine vision, it is characterized in that, under off-line state, character in image is split, and sort out structure character repertoire, adopt the convolutional neural networks learning method after improving to train, form stable sorter; Captured in real-time picture in ONLINE RECOGNITION, separating character, and classify with sorter; Specifically comprise following step:

(1) MSER process is carried out to pop can tank base map picture, orient character zone by morphology area-method;

(2) for the feature of dot character, adopt dot character segmentation module to split region, obtain single separating character;

(3) in character recognition, adopt convolutional neural networks learning method training character, form character classifier.

2. a kind of based on coding character detection method at the bottom of the pop can tank of convolutional neural networks according to claim 1, it is characterized in that: described step (1) specifically comprises the following steps:

(1) MSER algorithm process is adopted to the image obtained, uses different gray thresholds to obtain character zone to image binaryzation:

(1.1) adopt BinSort algorithm to given image, by the gray-scale value sequence in image, time complexity is 0 (n);

(1.2) extremal region merges and chooses, and according to descending or ascending order rule, the gray-value pixel after sequence is put into image, adopts Component Tree method to obtain each extremal region;

(1.3) on the Component Tree formed, from certain node, branch is searched for, according to the relative change rate in formulae discovery region in tonal range, in the branch of this Component Tree, try to achieve local minimum, obtain stable MSER region, be the character zone of coarse positioning;

(2) adopt 3*3 rectangular configuration to perform morphological dilation to image, and each connected component labeling in image is added up the area of connected domain; Arrange character zone attribute, the connected domain of area between (s1, s2) is character zone, and concrete numerical value adjusts according to different model pop can;

(3) rotate character zone, calculated the anglec of rotation of character zone by minimum enclosed rectangle, and adopt affined transformation and linear interpolation that the character zone of inclination is corrected to horizontal direction.

3. a kind of based on coding character detection method at the bottom of the pop can tank of convolutional neural networks according to claim 1, it is characterized in that: described step (2) specifically comprises the following steps:

(1) character row segmentation

(1.1) first adopt sciagraphy to the capable projection of character zone, add up its projection value Prover [i]:

$\mathrm{Prover}\left[i\right]=\underset{j=1}{\overset{\mathrm{row}}{\mathrm{\Σ}}}I(i,j)$

Wherein row is the line number of image;

(1.2) pop can character is dot character, often usually there is gap between row, the basis of projection localization adopts bellow expansion method, when adding up the projection value at every bit place, if by this point centered by, containing impact point in the scope being bound threshold value with bellow expansion number of times, then the projection value of these row is added 1, after selecting bellow expansion, the i-th projective representation being listed in horizontal direction is:

$\mathrm{ProVer}\left[j\right]=\underset{i=1}{\overset{\mathrm{low}}{\mathrm{\Σ}}}\mathrm{DilateGray}(i,j)$

$\mathrm{DilateGray}(i,j)=\left\{\begin{array}{ccc}1& \mathrm{if}& \underset{m=j-\mathrm{SEC}}{\overset{j+\mathrm{SEC}}{\mathrm{\Σ}}}I(i,m)\≠0\\ 0& \mathrm{if}& \underset{m=j-\mathrm{SEC}}{\overset{j+\mathrm{SEC}}{\mathrm{\Σ}}}I(i,m)=0\end{array}\right.$

Wherein I (i, j) represents the pixel value at bianry image (i, j) place, and SEC represents bellow expansion number of times;

(1.3) segmentation threshold is set: according to the image after bellow expansion, segmentation threshold threshold_row is set, when projection value is less than threshold_row after bellow expansion, be row split position;

(2) character row segmentation

(2.1) adopt vertical projection method, the overall width W in significant character region can be counted, according to the character number n of priori, calculate the mean breadth x of character;

(2.2) the bellow expansion number of times of row character is set, carries out bellow expansion; Segmentation threshold threshold_col is set, carries out coarse segmentation; Often row dot character interval is less, has partial character to be sticked together after bellow expansion, adopts conglutination segmentation method to split:

1. 2 Characters Stucks: all subregion character duration be partitioned into and average character duration x are compared, group peak width is greater than 1.3 times of average character duration, when being less than 2.2 times, is judged as two Connection operators; Again from initial split position, find out the minimum value of waveform projection value between distance split position 0.8x to 1.2x, the column position corresponding to it is split position;

2. 3 Characters Stucks: all subregion character duration be partitioned into and average character duration x are compared, group peak width is greater than 2.3 times of average character duration, when being less than 3.2 times, is judged as three Connection operators; From initial split position, find out between distance split position 0.8x to 1.2x and between 1.8x to 2.2x, the minimum value of waveform projection value, two column positions corresponding to it are the position of twice segmentation.

4. a kind of based on coding character detection method at the bottom of the pop can tank of convolutional neural networks according to claim 1, it is characterized in that: described step (3) specifically comprises the following steps:

Convolutional neural networks training algorithm is mainly divided into three phases: build network structure, propagated forward stage and back-propagation phase;

The first step, structure network structure:

The convolutional neural networks that the present invention builds adopts 7 Rotating fields, and the initial stage of training builds network structure, and adopt different little random numbers to carry out initialization to the weights of network, generally in [-1 ~ 1] scope, biased initialization is set to 0;

Second step, propagated forward stage:

(1) input layer input target image Y, and the target vector d of correspondence, convolutional layer carries out convolution by convolution kernel successively to input picture, and adds upper offset, then is obtained by nonlinear activation function:

$Y_j^n=f(\underset{i=1}{\overset{S}{\mathrm{\Σ}}}{Y}_i^{n-1}*W_{\mathrm{ij}}^n+{\mathrm{\φ}}_j^n)$

F (*) is RELU function:

$f\left(x\right)=\max (x,0)=\max (\underset{i=1}{\overset{S}{\mathrm{\Σ}}}{Y}_i^{n-1}*W_{\mathrm{ij}}^n+{\mathrm{\φ}}_j^n,0)$

(2) mode of down-sampling adopts stochastic pooling to sample, and formula is:

$Y_t={\mathrm{\Σ}}_{j\∈R_t}{p}_j{Y}_j$

Wherein: r _tfor the window size of layer of sampling, be generally 2*2 size, Y _jfor the element value of sample window;

(4) full attended operation is carried out to the characteristic pattern after convolution, down-sampling, and the actual output Q of computational grid layer F6 _k:

$O_k=f(\underset{t=1}{\overset{l}{\mathrm{\Σ}}}{V}_{\mathrm{tk}}{Y}_t+{\mathrm{\θ}}_k)$

3rd step, back-propagation phase

Back-propagation phase adopts gradient descent method to adjust weights and threshold, and statistics total error function is: as E≤ε, training terminates, and preserved by weights and threshold, at this moment network structure parameters is stablized, and sorter is formed.