CN108229483A

CN108229483A - Based on the doorplate pressed characters identification device under caffe and soft triggering

Info

Publication number: CN108229483A
Application number: CN201810044470.6A
Authority: CN
Inventors: 赵储; 李子印
Original assignee: China Jiliang University
Current assignee: China Jiliang University
Priority date: 2018-01-11
Filing date: 2018-01-11
Publication date: 2018-06-29

Abstract

The present invention proposes a kind of doorplate pressed characters identification device based under caffe and soft triggering, including hardware collecting device module and software handler module.Hardware collecting device module completes the acquisition of Pressed Character.The acquisition and acquisition of image are completed including camera acquisition module；And its type is distinguished, improve product information；Simultaneously Image Acquisition is carried out by software triggering photo module.Software handler module carries out image procossing, character cutting and identification work to the image of acquisition.First, whether detection image includes the circle of predetermined size, and the image for meeting condition is acquired, and avoids acquisition image definition too low；Secondly, it proposes that a kind of unrestrained water filling algorithm determines the position of character, eliminates jamming pattern；Finally, caffe is called to identify network with 5 hand-written scripts of LeNet, it is learnt and is identified, compared to traditional characteristic extraction algorithm, accuracy rate is significantly increased.

Description

Based on the doorplate pressed characters identification device under caffe and soft triggering

Technical field

The present invention relates to the technical fields such as image procossing and character recognition, and in particular to based under caffe and soft triggering Doorplate pressed characters identification device.

Background technology

Great demand is industrially suffered from numerous production lines for the identification device of doorplate pressed characters, it is existing It is following with being primarily present in terms of identification in Image Acquisition and character processing towards the coining doorplate character recognition device in industry The deficiency of several aspects, has much room for improvement：

First, common coining doorplate character recognition device is concentrated mainly in terms of image acquisition and triggering are taken pictures Image Acquisition work is carried out using hardware trigger image capture module, since hardware is there is mounting condition limiting factor is excessive, Such as：The limitation of the conditions such as place, environment, light, therefore tend not to be well adapted for various production scenes, while hardware touches Hair, without any selection, equally acquires preservation for the bad picture of image quality, causes in terms of the picture quality of acquisition The wasting of resources and recognition efficiency, the reduction of discrimination；Secondly usual means can not make in product category the shop front and doorframe Distinguish, can be only done simple counting and camera function, for product category needed for subsequent production process and sequence number without Method provides corresponding data source, and equipment applicability is not high, therefore in recent years for the kind equipment in image acquisition and its triggering side There is a degree of optimization in formula and improve.

Secondly, common pressed characters identification device is mainly traditional using being based on character identifying method on the market at present Template characteristic matching and the algorithm based on structures statistics, such as：Using the gray scale of character picture as match information, pass through meter It calculates Character mother plate image and the gray scale related coefficient of target image subregion to be identified carrys out metrics match degree；Or utilize character sheet The structural form of body, such as edge, inflection point, connected region feature carry out pattern-recognition；Or in the feature for establishing character picture Later, the relationship by way of having supervision between learning characteristic and target classification, so as to establish from the function for being input to output Relationship.Identification technology based on traditional template matches and feature extraction, character recognition effect in ideal circumstances are preferable；But It is the complexity diversification with production environment, on the one hand artificial character feature of choosing often is difficult to find that most suitable feature, separately Character recognition effect of the one side conventional method under complex scene is bad, it is difficult in real time and accurately identify doorplate character. In recent years, with neural network and machine learning rise and development and relevant hardware devices condition it is perfect, utilize god Data training is carried out through network and deep learning frame, allows computer autonomous learning, the feature of data to be tested is extracted, establishes and know Other model becomes the popular skill industrially identified at present for multi-target detection so as to achieve the effect that autonomous classification Art.

The present invention identifies dress in view of the above problems, proposing using based on the doorplate pressed characters under caffe and soft triggering It puts, carries out the identification of doorplate pressed characters.First, the type of product where doorplate is distinguished based on infrared shooting device, Facilitate the follow-up work in production process, further improve product information, improve the applicability of device；Secondly, it is touched using soft Originating party formula carry out Image Acquisition, avoid by image definition caused under hardware trigger it is not high the problem of, be so as to improve System Image Acquisition ability；Finally, hand-written script identification deep learning frame LeNet-5 is relied on, by calling in caffe frames LeNet-5 hand-written scripts identification network frame, improve the accuracy of doorplate character recognition algorithm, robustness and calculate speed Degree, can better adapt to current production requirement, there is stronger applicability in industrial production.

Invention content

The present invention is first by hardware devices such as camera acquisition module, infrared emission module and industrial personal computers, by soft The mode of part triggering completes the acquisition of character picture and the differentiation work of product category where doorplate character；Secondly unrestrained water is utilized Filling algorithm handles character picture region, removes background interference, obtains complete and single character zone image, completes Positioning work to character；Network frame is identified using based on the LeNet-5 hand-written scripts in caffe frames, carry out word simultaneously The training and study of symbol, realize the identification function for doorplate pressed characters.The invention is realized by the following technical scheme：

Based on the doorplate pressed characters identification device under caffe and soft triggering, handled including hardware acquisition module and software Program module two parts.The hardware acquisition module includes camera acquisition module, infrared emission module, collection control module；Institute The software handler module stated includes software triggering photo module, image pre-processing module, character cutting module, caffe Habit and identification module；The camera acquisition module includes camera 1 and camera 2；The infrared emission module Including infrared emission module 1 and infrared emission module 2, infrared emission module 1 and infrared emission module 2 be divided into not by To penetrating transmitting terminal with being formed to penetrating receiving terminal.

Based on the doorplate pressed characters identification device under caffe and soft triggering, described in hardware acquisition module Camera acquisition module, infrared emission module and collection control module placement position and its connection mode be described as follows：First, Camera acquisition module is mainly responsible for acquisition current character image, answers pickup area designated position demand on production line, to camera shooting First No. 1, the placement position of No. 2 do it is as described below：Remember camera 1 corresponding to can shooting area range ultra-Left side with camera shooting Corresponding to first No. 2 can shooting area range the intersection point that is formed of ultra-Left side for intersection point a, remember clapping corresponding to camera 1 Take the photograph corresponding to the ultra-Right side of regional extent and camera 2 can the intersection point that is formed of ultra-Right side of shooting area range be intersection point b, Camera 1 need to ensure that intersection point a and intersection point b is respectively positioned on doorplate production line and passes with No. 2 placement locations in production line of camera It send in direction of advance, under this placement location, it is ensured that when product is by production line direction of transfer, be accurately completely located at The visual field central area of camera is imaged more complete display at this time, and character content is completely obtained, and is not broken scarce The situation of mistake；Second, since pressed characters are located at door or doorframe edge both sides, infrared emission module 1 should be located at intersection point a Left side, and should be less than with the distance of intersection point b the width of a doorframe；Infrared emission module 2 is located on the right side of intersection point b, and with friendship The distance of point a should be less than the width of a doorframe；Ensure that the only shop front can be infrared right to one of which when triggering is taken pictures with this Module generation is penetrated to block；Third, the collection control module are connect with camera acquisition module by usb data connecting line, transmission Data, collection control module are then connected with infrared emission module by capture card with serial port connector, transmit signal.

Based on the doorplate pressed characters identification device under caffe and soft triggering, operation principle is as follows with flow：

First, character picture is acquired by hardware acquisition module, wherein camera acquisition module is mainly responsible for acquisition image； Infrared emission module judges product category；Collection control module is responsible for the processing and preservation of operation, signal transmission and image.

Secondly, by software handler mould software in the block trigger photo module complete regional choice that image is imaged with It takes pictures；Image pre-processing module is to the processing of character picture, location character position；After character cutting module is to positioning Training character, test character and character to be detected carry out cutting, form corresponding character data collection；Caffe learns and identification mould Training and study of the block to character data collection, complete last identification work.

Now the operation principle of modules in device is described in detail：

Product is logical first when being transmitted through production line via camera acquisition module and infrared emission module where doorplate It crosses the triggering of the software described in software handler module photo module to be detected, judges currently to acquire to whether there is in picture to meet Be sized size it is round exist (it should be noted that：Have one in this regulation acquiescence coining doorplate character zone to be identified Coining is round, and inside coining has the character of representative products grade, and its size is fixed, can be as the screening mark to product It is accurate), when detecting existence anduniquess in current acquired image and clearly meeting the circle for being sized size, illustrate currently to adopt It is effective to collect region, then current character picture is acquired by way of soft triggering and taken pictures, while detect infrared emission mould The on off operating mode of block 1 and infrared emission module No. 2 numbers：When infrared emission module 1 and infrared emission module 2 are simultaneously conducting During state, product category is doorframe where judging current doorplate character；Other states judge product where current doorplate character Type is the shop front, so as to be distinguished to product category, while the image currently acquired is preserved and recorded, for doorframe And the shop front, after type is differentiated, for the convenient intuitive differentiation of user, on the third position of filename name, by doorframe 1 is denoted as, the shop front is denoted as 0, is distinguished to show, more complete product data information is provided in the production process after being.

The triggering mode of taking pictures that the present invention is used in image acquisition phase is soft triggering, compared to hardware trigger shooting style For, soft triggering is of less demanding to the hardware condition of environment where device and surrounding, and need not install hardware device, flexibly Property is stronger, by whetheing there is the detection of standard circular to doorplate character zone, further defines the range of camera collection image, For image quality is bad, circle detection size or clarity are undesirable, there is no required words to be identified in imaging region The image for according with region not acquires, and both reduces acquisition range, also reduces the calculation amount of subsequent image processing work, improves The processing speed of device and applicable ability.

First, for the character picture collected, first through the image preprocessing mould described in software handler module Block completes the accurate positionin of the processing and character position to character picture, obtains the pure character figure without background interference Picture.The specific processing step flow of the module is as follows：

Step 1：Image gray processing is carried out first for the character picture collected, is reducing image processing data amount While ensure image in essential information do not lose；Secondly low pass gaussian filtering and medium filtering are carried out, to making an uproar in image Sound point is tentatively removed, and further improves picture quality；Later using sobel operators to carrying out gradient fortune in image X-direction It calculates, (is only herein to carrying out gradient algorithm reason in image X-direction：The purpose for carrying out edge detection herein is tentatively to obtain The location information of character is taken, and character position concentration is stamped on the vertical direction of doorplate, therefore only according to priori Need the gradient algorithm to image progress X-direction)；Image binaryzation is carried out again, and character picture is converted into black and white two-value Image removes part background, obtains the preliminary profile of character zone；Form student movement finally is carried out to the image after binaryzation It calculates, eliminates the tiny region of fracture in character and tiny noise spot, short and small pixel is attached merging, thus into one The interference in step removal non-character region.

Step 2：For by the character picture after step 1, carrying out the detection and positioning of doorplate character outline：First By the contour area of edge contour search function, substantially searching character image, the position coordinates matrix on profile boundary is returned Contours, and then ask for its minimum enclosed rectangle；Secondly character is removed using unrestrained water filling algorithm to the boundary rectangle of acquisition Non-character region in image background only retains doorplate character zone.

Step 3：For by the character picture after step 2, being done using edge detection Canny operators to character picture Edge detection obtains the substantially edge contour of character picture；Secondly, by meeting particular requirement in Hough transformation function check figure Straightway.It should be noted that according to priori, character region is located substantially at the 1/3 to 2/3 of entire image Region, therefore 1/3 to 2/3 region that detection zone is entire image is determined first, for drawing out utilization successively in the region Every line segment that Hough transformation function check arrives is unsatisfactory for the line segment essentially like vegetarian refreshments length requirement for length, is screened.

Length is only retained after Hough transformation detection of straight lines, in image more than certain number of pixels and two straight lines Between interval it is straight more than given threshold (threshold value can adjust based on experience value herein, and 20 pixels are set as in this algorithm) Line segment, and obtained straight-line segment is finally detected to these and is ranked up according to coordinate position, according to straight line position, sequence X, the Y coordinate of the Origin And Destination of every straight line are returned, according to starting and terminal point coordinate, may finally determine character region Boundary line is denoted as image dst, may finally obtain the determining position of doorplate character, thus completes the detection of character outline with determining Position work.

Step 4：For by the character picture dst after step 3, first two straight line of calculating character image up-and-down boundary Slope, be denoted as K1 and K2 respectively, for K1 and K2, seek its mean value, be denoted as K, and then calculate rotation angle；Secondly, with word The center point coordinate of image-region is accorded with as rotation center, the image dst2 after character picture dst is rotated, Purpose is character picture rotation horizontally, to facilitate subsequent character cutting and identification.

After more than processing step, pure character picture region complete and without background interference factor, and word can be obtained Symbol imprint sequence meets common character order, and direction is arranged for horizontal direction, and the character cutting module for after provides figure As data.

2nd, it to the character picture obtained by above-mentioned steps, by character cutting resume module, obtains single after cutting Character picture, and cutting result figure of the same size is standardized as, result figure will be in follow-up identification module as training number According to source.The specific processing step flow of the module is as follows：

Step 1：First, to character picture dst2, using the upper left corner of image as the starting point of pixel search, for whole Picture, from top to bottom progressive scan judge the pixel value of every bit in image, and main process flow is as follows：

1. judging whether the pixel value is 0, if the pixel value is 0 (representing that the point is black), continue to scan on down Some pixel values；If the pixel value is 1 (representing that the point is white), judgement 2 is carried out.

2. judging whether the ordinate is between 1/3 to the 2/3 of picture overall width, ordinate is unsatisfactory for will The then return Rule of judgment 1 asked, carries out the judgement of next point；If meeting the requirements, judgement 3 is carried out.

3. on the basis of taking the ordinate, the point into each 30 pixel coverages of from left to right counts the picture put in the range of this Element value is 1 pixel number, judges whether the pixel number met the requirements is more than the region all pixels point number 80%, if being unsatisfactory for requiring, Rule of judgment 1 is returned, carries out the judgement of next point；The seat of the point is returned if meeting the requirements Scale value is denoted as point P1 (X1, Y1).

Step 2：Secondly, to character picture dst2, using the lower left corner of image as the starting point of pixel search, for whole Picture, from bottom to up progressive scan judge the pixel value of every bit in image, and main process flow is the same as the place in step 1 Flow is managed, thus obtained point coordinates value is denoted as P2 (X2, Y2).

Step 3：To character picture dst2, using the upper left corner of image as the starting point of pixel search, for whole pictures, The pixel value for judging every bit in image is scanned by column from left to right, and main process flow is the same as the processing stream in step 1 Journey, thus obtained point coordinates value are denoted as P3 (X3, Y3).

Step 4：To character picture dst2, using the upper left corner of image as the starting point of pixel search, for whole pictures, The pixel value for judging every bit in image is scanned by column from right to left, and main process flow is the same as the processing stream in step 1 Journey, thus obtained point coordinates value are denoted as P4 (X4, Y4).

Step 5：The coordinate value of four points obtained according to step 1 to step 4, character is obtained according to the following formula Specific location confines rectangle frame, further reduces character position location：

Define the width and length that height and width is respectively rectangle, note：

Height=| Y2-Y1 |

Width=| X4-X3 |

A rectangle frame is confined on image dst2 in this way on artwork character picture and after binaryzation, it is believed that the rectangle Frame is the accurate outer rim of character.

Step 6：To the character picture after determining accurate outer rim, need further to obtain single character cutting knot Fruit is schemed, and main process flow is as follows：

1. for the character picture after determining accurate outer rim, first according to priori, divide chinese character：According to The length and width data (W3, H3 are the width and length of chinese character in pressed characters model according to priori) of W3, H3, Using W3 as standard on character picture, the substantially right margin of first character, i.e. chinese character is first determined；Again with the big of chinese character Cause right margin is starting point, scans by column character picture from right to left, traverses the pixel value in each row character picture, and if only if The pixel value be 1 (representing that the point is white) and on the basis of taking the abscissa, upwards downwards in each 30 pixel coverages Point, counts the pixel number that the pixel value put in the range of this is 1, and the pixel number met the requirements is more than all pictures in the region Vegetarian refreshments number 80% when, record the abscissa of current point, using the abscissa as axis, place straight line is the accurate of chinese character Right margin.Thus it can obtain the exact position of first character, i.e. chinese character.

2. for 6 characters that letter later and digital random are formed, also according to W1, H1, W2, H2 is (assuming that number The character boundary of word 0-9 templates is W1, H1；Alphabetical A-Z template sizes be W2, H2) size width, first determine character substantially Right margin, then character picture is scanned by column from substantially right margin right-to-left, the pixel value in each row character picture is traversed, into And determine the accurate right margin of character, the exact position of each character is thus obtained successively.

3. for the single character after Accurate Segmentation, finally by the picture that its size normalization is 28*28, because subsequently Network model requires its input data size as 28*28, will be denoted as cutting result by the single character after size normalization Figure.Cutting result figure is stored according to the content of corresponding single character under corresponding file, character content and folder name It corresponds.

It should be noted that：

1. the corresponding doorplate pressed characters of the device, character are imprinted by template, character mainly have number, letter with And " the first and second the third fourth " four class Chinese characters are formed, the character impression block size of the same category is unified, i.e., all digital template sizes, All alphabetical template size and the template size of the first and second the third four kinds of Chinese characters of fourth are identical, but size is not between these three characters Together.

2. due in pressed characters, alphabetical I is difficult to differentiate between with number 1, letter O and number 0, therefore, device acquiescence pressure Without letter I and O in lettering symbol.

3. and according to priori, it is assumed that the character boundary of digital 0-9 templates is W1, H1；Alphabetical A-Z template sizes are W2, H2；The the first and second the third fourth template size of Chinese character is W3, H3.The size of chinese character is maximum, and the ruler of letter and number character Very little size is identical, is slightly less than chinese character.

4. the queueing discipline of doorplate pressed characters from left to right is：From left to right first character be Chinese character, subsequent words It accords with and being formed at random for number and letter, altogether 7 characters.

5. cutting result figure is known as training cutting result in follow-up identification module as training data source, training network Figure；As test data source, the good and bad referred to as test cutting result figure of test network in follow-up identification module；Subsequently identifying It is known as cutting result figure to be identified as character data collection to be identified in module.

6. the picture number obtained according to actual acquisition, using 60,000 character figures therein as training cutting result figure Data source, it is another using data source of 10,000 collected character figures as test cutting result figure, it should be noted that herein 60000 training sample sources are independent misaligned mutually with 10,000 test specimens origins.

7. being fixed as except Chinese character since doorplate pressed characters remove initial character, the type and content of follow-up 6 characters are equal It randomly generates, therefore cannot be guaranteed the cutting result figure uniform amount of number and letter distribution under variety classes, for a If the character cutting result figure quantity under other type is very few, may be used basic image processing method (such as：It rotates, go Noise, fuzzy, morphological operation etc.) generation, and then completion cutting result figure.

According to above-described six steps, the single character after it can finally obtain cutting after character cutting module Image result, and be stored under corresponding file according to character content one-to-one correspondence, corresponding cutting result figure is formed, It prepares for character recognition later.

Four, identify the cutting result figure obtained by character cutting resume module through the hand-written script in caffe frames Network, carries out the study of network model, and and then test the quality of current network model, finally character to be identified and training are learned Acquistion to network be combined, complete the identification work of last character content.

Caffe learns to include study module and identification module, the wherein specific process flow of study module again with identification module (involved file path name acquiescence is since caffe roots herein) as follows：

Step 1：Before the device carries out character recognition, need to learn network model, to there is a large amount of sample Input, therefore for the advanced row manual sort of training cutting result figure, corresponding character is saved in the file of corresponding name respectively Under folder, training sample set is formed.

Step 2：Manual sort is carried out for test cutting result figure, corresponding character is saved in corresponding name respectively Under file, test sample collection is formed.

Step 3：The picture that training sample is concentrated is tagged in sequence, upsets sequence, forms label file, and Path where recording label file, the picture concentrated for test sample do identical operation, generate corresponding label file.

Step 4：Picture format is converted.Since the data format that input is required in LeNet-5 networks is ldb or lmdb Form, therefore, it is necessary to carry out the conversion of picture format.Format conversion is as follows：

1. the cpp files of picture format conversion and corresponding exe tools are come under caffe roots path.

2. under caffe roots path, create_mnist.bat files (windows batch operations file) are created simultaneously Content therein is made an amendment.

After 3. modification finishes, run bat files, the respective path that will be preserved after being disposed in the needs of setting Under, generate corresponding test set and training set file.

Step 5：Parameter in LeNet-5 networks and path where file under corresponding modification caffe parcels.Step 6： Bat files are run, generate lenet_iter_10000.caffemodel models.

Step 7：Test network model is good and bad.In order to which the network model of gained in testing procedure six is non-training for other How is picture recognition effect in sample set, and the picture concentrated using test sample tests network.It should be noted that 60000 training samples and 10,000 test samples are mutual independent misaligned, therefore, can test out the network to different numbers According to good and bad performance, this step is also the covert quantity for increasing training sample, has better training effect for network.Root According to the experiment result understand the model it is very high for the character image data accuracy rate in test set and will not occur over-fitting and It is absorbed in the cycle of local optimum.

After processing by above step one to step 7, you can obtain the model file after training optimization Caffemodel will subsequently utilize it, character picture to be identified will be identified.

Five .caffe study is as follows with the specific process flow of the identification module in identification module：Step 1：It is transported in device During row, based on hardware collecting device module collected character picture to be identified, also pass through above-mentioned flow and obtain Cutting result figure to be identified.

Step 2：By the cutting result figure to be identified of gained in step 1 according to caffe study and in identification module It practises step 4 in module to be handled, obtains the data set after format conversion, be denoted as character data collection to be identified.

Step 3：Caffe study and the model file caffemodel obtained by identification module learning module are called, will be treated Identification character data, which is put into model, to be identified, and final output is with comparing the highest corresponding word of similarity probability value in model library Symbol, the i.e. recognition result of current character, then be combined output, as final doorplate pressed characters recognition result are thus complete Into final character recognition.

Advantages of the present invention：

1st, the present invention relies on using caffe deep learnings network as calculating instrument and calls LeNet-5 convolutional neural networks Convolution algorithm is carried out, in a manner that training dataset and test data set separately train study.Training dataset training generation Model file, the model of test data set pair training generation are tested, and used test data set and training dataset are only It is vertical to separate so that model met more discrepant data in the training process, so as to further improve the robustness of model And generalization ability, also there is preferable robustness for real data.

2nd, training data speed of the present invention is fast, and algorithm high degree of automation is easy to use, efficient.

3rd, the present invention is taken pictures using soft triggering mode triggering video camera and acquires image, is avoided and is drawn under hardware trigger mode The problem of image definition risen is not high, is filtered screening, so as to improve system diagram for the poor picture of image quality As acquisition capacity.

4th, hand-written script convolutional network framework LeNet-5 is applied to the character recognition in safety door production process by the present invention In demand, LeNet-5 networks are promoted and applied.

5th, the present invention distinguishes label by infrared emission to the type of product where doorplate, and correspondence image also carries out area It does not preserve, facilitates the follow-up work in production process, further improve product information, further improve the applicability of device.

6th, the present invention employs a kind of unrestrained water filling algorithm to obtain the position of character during location character position Confidence ceases, and accurately completely eliminates the background area of picture, so as to improve the character extractability of device, makes that it is suitable for more Under mostly more complicated background environment.

Description of the drawings

Fig. 1 device overall structure diagrams；

Fig. 2 device hardware collecting device module mounting location schematic diagrames；

Fig. 3 device camera acquisition module imaging schematic diagrams；

Fig. 4 device softwares trigger photo module flow chart；

Fig. 5 device image pre-processing module flow charts；

Fig. 6 device character cutting module flow diagrams；

Fig. 7 device doorplate surface structure schematic diagrams；

Specific embodiment

Patent of the present invention is further described with case study on implementation below in conjunction with the accompanying drawings, but is not intended as special to the present invention The foundation of profit limitation.

As shown in Fig. 1 device overall structure diagrams, the present apparatus is by hardware collecting device module 1 and software handler mould 2 two parts of block form, and wherein hardware collecting device module 1 includes camera acquisition module 3, infrared emission module 4, collection control module 5；Software handler module 2 includes software triggering photo module 6, image pre-processing module 7, character cutting module 8, caffe Study and identification module 9；Camera acquisition module 3 is again including camera No. 1 10 and camera No. 2 11；Infrared emission module 4 is wrapped Include infrared emission module No. 1 12 and infrared emission module No. 2 13, infrared emission module No. 1 12 and infrared emission module No. 2 13 are again Respectively by penetrating transmitting terminal and being formed to penetrating receiving terminal, collection control module 5 passes through usb data connecting line with camera acquisition module 3 Data are transmitted in connection, and collection control module 5 is then connected with infrared emission module 4 by capture card with serial port connector, transmits letter Number.

Based on the 16 pressed characters identification device of doorplate under soft triggering first by being based on camera collecting device 3, infrared The acquisition of character picture is completed with being produced where 16 character of doorplate to penetrating the hardware devices such as module 4 and collection control module 5 (industrial personal computer) The differentiation work of kind class；Secondly character picture is pre-processed, character picture region is carried out using unrestrained water filling algorithm Processing, removes the interference of background pixel, obtains complete and single character picture, complete the positioning work to pressed characters；Together Shi Caiyong is based on caffe deep learnings and identifies network frame with LeNet-5 hand-written scripts, carries out the training and study of network, most The identification function for 16 pressed characters of doorplate is realized eventually.

In the device operational process, as Fig. 2 device hardware collecting device module mounting locations schematic diagram, Fig. 3 devices image Head acquisition module imaging schematic diagram, Fig. 4 device softwares triggering photo module flow chart and the signal of Fig. 7 device doorplates surface structure Shown in figure, according to the hardware placement position between camera acquisition module 3 and infrared emission module 4, when coining 16 character of doorplate Place product is transmitted by production line, by camera acquisition module 3, into camera No. 1 10 and camera No. 2 11 During imaging region, need to ensure current 16 pressed characters of doorplate in the imaging region of camera No. 1 10 and camera No. 2 11 Centre：It changes in triggering infrared emission module 4 to the state for penetrating pipe first, secondly, passes through institute in software handler module 2 The software triggering photo module 6 stated is detected, and is judged to whether there is in current acquisition picture and is met the circle for being sized size and deposit When detecting existence anduniquess in current acquisition picture, clear and when meeting the circle for being sized size, illustrate current acquisition Region is effective, then current character picture is acquired by collection control module 5 by way of soft triggering and taken pictures；It detects simultaneously red Outside to penetrating the on off operating mode of module No. 1 12 and infrared emission module No. 2 13：When infrared emission module No. 1 12 and infrared emission mould Block No. 2 13 simultaneously for conducting state when, product category is doorframe 14 where judging current 16 character of doorplate；Other states judge Product category where 16 character of current doorplate is the shop front 15, so as to be distinguished to product category, and the image to currently acquiring It is preserved and is recorded, for doorframe 14 and the shop front 15, after type is differentiated, intuitively distinguished in order to which user is convenient, On the third position of filename name, doorframe 14 is denoted as 1, the shop front 15 is denoted as 0, is provided in the production process after being more complete Product data information.

16 character picture of coining doorplate collected through hardware collecting device module 1 is transferred first through collection control module 5 To character picture, through the image pre-processing module 7 described in software handler module 2, complete to add the processing of character picture The accurate positionin of work and character position obtains the pure character picture without background pixel interference.Such as Fig. 5 device image preprocessings Shown in module flow diagram, the module main processing steps are as follows：

Step 1：Image gray processing is carried out first for the character picture collected, is reducing image processing data amount While ensure image in essential information do not lose；Secondly gaussian filtering and medium filtering are carried out, to the noise spot in image It is tentatively removed, further improves picture quality；Later using sobel edge detection operators to carrying out ladder in image X-direction Spend operation；Image binaryzation is carried out using Threshold segmentation later, character picture is converted into black and white binary image, removal part is carried on the back Scape obtains the preliminary profile of character zone；Morphology operations finally are carried out to the image after binaryzation, are eliminated thin in character Short and small pixel is attached merging by the small region of fracture and tiny noise spot, so as to further remove non-character region Interference.

Step 2：Carry out the detection and positioning of 16 character outline of doorplate：First by edge contour search function function, greatly Cause the contour area of searching character image, return to the position coordinates matrix contours on profile boundary, so ask for it is therein most Small encirclement matrix；Secondly to the circumference matrix of acquisition, using unrestrained water filling algorithm, the character picture back of the body can preferably be removed Non-character region in scape, by setting can connected pixel bound and mode of communicating achieve the effect that filling, so as to Only retain 16 character zone of doorplate.

Step 3：Edge detection is done to character picture using edge detection Canny operators, obtains the substantially side of character picture Edge profile；Secondly, by meeting the straightway of particular requirement in Hough transformation function check figure, detection zone is entire image 1/3 to 2/3 region, for drawn out successively in the region using Hough transformation function check to line segment screen, for Length is unsatisfactory for giving up essentially like the line segment of vegetarian refreshments length requirement.After Hough transformation detection of straight lines, only retain in image Lower length is more than interval between certain number of pixels and two straight lines, and more than given threshold, (threshold value can be done based on experience value herein Adjustment is set as 20 pixels in this algorithm) straightway, and obtained straight-line segment is finally detected according to coordinate bit to these It puts and is ranked up, according to straight line position, sequentially return to X, the Y coordinate of the Origin And Destination of every straight line, according to starting point end Point coordinates may finally determine the boundary line of character region, be denoted as image dst, may finally obtain 16 character of doorplate Determine position.

Step 4：To the slope of character picture dst, first two straight line of calculating character image up-and-down boundary, it is denoted as K1 respectively, K2 for K1, K2, seeks its mean value, is denoted as K；Secondly, using the center point coordinate in character picture region as rotation center, by character Image dst is rotated, and calls rotation function, and wherein parameter degree, i.e. rotation angle seek its triangular transformation by K, is calculated Its corresponding angle obtains, and the image after being rotated is denoted as image dst2, it is therefore intended that character picture is rotated into level Direction, guarantee character are horizontal direction, facilitate subsequent character cutting, then carry out image preservation.

After more than processing step, pure character picture region complete and without background interference factor, and word can be obtained Symbol imprint sequence meets common character order, and direction is arranged for horizontal direction, and the character cutting module 8 after being provides figure As data.

Obtained character picture is handled through the image pre-processing module 7 described in software handler module 2, as Fig. 6 is filled It puts shown in character cutting module flow diagram, to the character picture after process image pre-processing module 7, first with the upper left of image Starting point of the angle as pixel search, for whole pictures, progressive scan from top to bottom judges the pixel value of every bit in image：It is first First judge whether the pixel value is 0, if the pixel value is 0 (representing that the point is black), continue to scan on subsequent point pixel Value；If the pixel value is 1 (representing that the point is white), judge whether the ordinate is in the 1/3 of picture overall width To between 2/3, the then return Rule of judgment 1 of requirement is unsatisfactory for for ordinate, carries out the judgement of next point；It will if meeting It asks, then carries out judgement 3：On the basis of taking the ordinate, the point into each 30 pixel coverages of from left to right counts point in the range of being somebody's turn to do Pixel value be 1 pixel number, judge the pixel number met the requirements whether be more than the region all pixels point number 80%, if being unsatisfactory for requiring, return to Rule of judgment 1, carry out the judgement of next point；The point is returned if meeting the requirements Coordinate value is denoted as point P1 (X1, Y1).It is same to judge that flow from top to bottom, is scanned, obtained for whole pictures difference from left to right To the coordinate value of four points of accurate character position, it is denoted as P2 (X2, Y2)；P3 (X3, Y3)；P4 (X4, Y4)；Secondly, According to the coordinate value of four points, the specific location of character is obtained, confines rectangle frame, further reduce character position location； Later, to the character picture after determining accurate outer rim, according to priori, divide single character successively.First, divide the Chinese Word character：According to W3, the length and width data of H3 (width and length that imprint chinese character in model), with W3 on character picture For standard, the substantially right margin of first character, i.e. chinese character is first determined；Again using the substantially right margin of chinese character as rise Point, scans by column character picture from right to left, traverses the pixel value in each row character picture, is and if only if the pixel value 1 (represent the point for white) and on the basis of taking the abscissa, the point in each 30 pixel coverages downwards, counts the range upwards The pixel value of interior point is 1 pixel number, and the pixel number met the requirements is more than the region all pixels point number When 80%, the abscissa of current point is recorded, using the abscissa as axis, place straight line is the accurate right margin of chinese character.By This can obtain the exact position of first character, i.e. chinese character；Secondly, letter later and digital random are formed 6 characters, also according to W1, H1, W2, the size width of H2 first determines the substantially right margin of character, then from substantially right margin Right-to-left scans by column character picture, traverses the pixel value in each row character picture, and then determine accurate the right of character Thus boundary obtains the exact position of each character successively；For the single character after Accurate Segmentation, finally its size is returned One turns to the picture of 28*28 pixels, carry out herein size normalization purpose be it is follow-up used by network model requirement its Input data size is 28*28, cutting result figure, cutting result figure will be denoted as by the single character after size normalization Content according to corresponding single character is stored under corresponding file, and character content is corresponded with folder name.

According to step described above, the single character picture after it can finally obtain cutting after character cutting module 8 As a result, and be stored under corresponding file according to character content one-to-one correspondence, corresponding cutting result figure is formed, for it Character recognition afterwards provides data.

Handle obtained single character picture through the character cutting module 8 described in software handler module 2 as a result, through Caffe learns to identify network by caffe and LeNet-5 hand-written scripts with identification module, and the study of network model is completed in processing, And the quality of current network model is tested, finally character to be identified with the network that training study obtains is combined, is completed last Character content identification work, idiographic flow is as follows：

First, it before the device carries out character recognition, needs to learn network model, needs a large amount of sample defeated Enter, therefore for the advanced row manual sort of training cutting result figure, corresponding character is saved in the file of corresponding name respectively Under, training sample set is formed, manual sort is carried out for test cutting result figure, corresponding character is saved in corresponding name respectively File under, formed test sample collection.

Secondly, picture training sample concentrated is tagged in sequence, upsets sequence, forms label file, and remember Path where recording label file, the picture concentrated for test sample do identical operation, generate corresponding label file.

Later, carry out the conversion of picture format, due to required in LeNet-5 networks input data format be ldb or Therefore lmdb forms, in the final step of image data processing, need to carry out the conversion of picture format.

After modification finishes, bat files, the respective path that will be preserved after being disposed in the needs of setting are run Under, generate corresponding test set and training set file, included in be exactly training set and test after converting The data of collection.Later, bat files are run, generate model file.

The quality of test network model.The picture in other non-training sample sets is known in order to test gained network model How is other effect, network need to be tested using the picture that test sample is concentrated, it should be noted that 60,000 collected It is mutual independent misaligned to open training sample and 10,000 test samples, therefore, can test out the network to different data Good and bad performance.

Finally, in device operational process, based on hardware collecting device module 1 collected character figure to be identified Picture also passes through above-mentioned flow and obtains cutting result figure to be identified；Cutting result figure to be identified is learnt and identified according to caffe Study module in module 9 is handled, and obtains the data set after format conversion, is denoted as character data collection to be identified；It calls Caffe study and the model file lenet_iter_10000.caffemodel obtained by 9 learning module of identification module, will treat Identification character data, which is put into model, to be identified, and final output is with comparing the highest corresponding word of similarity probability value in model library Symbol, the i.e. recognition result of current character, are combined output, thus as final character identification result completes final word Symbol identification.

The foregoing is merely the preferred embodiments of patent of the present invention.

Claims

1. based on the doorplate pressed characters identification device under caffe and soft triggering, at hardware collecting device module and software Program module two parts are managed, the hardware collecting device module includes camera acquisition module, infrared emission module, collection control mould Block；The software handler module include software triggering photo module, image pre-processing module, character cutting module, Caffe learns and identification module；It is characterized in that judging whether to take pictures to present image by software detection trigger, avoid Image definition problem under hardware trigger form improves the Image Acquisition ability of device；Meanwhile pass through infrared emission mould Block auxiliary judgment goes out the shop front and the type of doorframe, further improves the applicability of device, is provided for subsequent production process More detailed product information, the data processing after facilitating work with product assortment；Secondly, using unrestrained water filling algorithm to figure As being pre-processed, to eliminate picture background, the character extractability of device is improved；Finally, by calling in caffe frames LeNet-5 hand-written scripts identify study and identification of the network into line character, and the model that training is obtained is applied to door to be identified Board Pressed Character is finally reached the purpose of identification character.

2. existed according to described in claim 1 based on doorplate pressed characters identification device its feature under caffe and soft triggering In camera acquisition module includes camera 1, camera 2；Infrared emission module includes infrared emission module 1, infrared To penetrating module 2；Infrared emission module No. 1 and No. 2 is respectively by penetrating transmitting terminal with being formed to penetrating receiving terminal；Remember camera 1 The ultra-Left side of shooting area is intersection point a with the intersection point that the ultra-Left side of No. 2 shooting areas of camera is formed, and note camera 1 is shot The ultra-Right side in region is intersection point b, camera 1 and camera 2 with the intersection point that the ultra-Right side of No. 2 shooting areas of camera is formed Placement location needs to ensure that intersection point a and intersection point b is located in doorplate production line transmission direction of advance；Infrared emission module 1 with it is red Outside to penetrating No. 2 outsides for being respectively placed in intersection point a and intersection point b of module；In addition, infrared emission module 1 is located on the left of intersection point a, And it should be less than the width of a doorframe with the distance of intersection point b；Infrared emission module 2 is located on the right side of intersection point b, and with intersection point a's Distance should be less than the width of a doorframe；Collection control module transmits data, collection control mould with camera acquisition module by USB data line Block is connect with infrared emission module by capture card with serial ports, transmits signal.

3. existed according to described in claim 1 based on doorplate pressed characters identification device its feature under caffe and soft triggering In software triggering photo module process flow is following steps：

Step 1：The doorplate pressed characters picture obtained in claim 1 by camera is whether there is using Hough transformation Circular detection.

Step 2：When detect in picture existence anduniquess and it is clearly round when, illustrate that current pickup area is effective, pass through software Triggering is acquired current character image and takes pictures, while detects the break-make of infrared emission module 1 and infrared emission module No. 2 numbers State：When the two is simultaneously conducting state, judge current production type for doorframe；Other states judge current production type For the shop front, so as to be distinguished to product category.

Step 3：The image currently acquired is kept records of.For doorframe and the shop front, after type is differentiated, in order to make The convenient intuitive differentiation of user, on the third position of picture file name name, 1 is denoted as by doorframe, the shop front is denoted as 0, is distinguished to show.

4. existed according to described in claim 1 based on doorplate pressed characters identification device its feature under caffe and soft triggering In image pre-processing module process flow is following steps：

Step 1：Gray processing is carried out first to the picture after processing preserves in claim 3, secondly carries out low pass gaussian filtering With medium filtering, the noise spot in image is tentatively removed；Later using edge detection sobel operators to image X-direction Gradient algorithm is carried out (herein only to carrying out gradient algorithm reason in image X-direction to be：The purpose for carrying out edge detection herein exists In the location information for tentatively obtaining character, and character position concentration is stamped in the vertical direction of doorplate according to priori On, therefore only need to carry out image the gradient algorithm of X-direction)；Image binaryzation is carried out using Threshold Segmentation Algorithm, it will Image is converted into black and white binary image, obtains the preliminary profile of character zone；Finally the image after binaryzation is successively carried out Morphology opening operation and closed operation eliminate the tiny region of fracture in character and tiny noise spot, short and small pixel are connected Merge, so as to further remove the interference in non-character region.

Step 2：For by the character picture after step 1, carrying out the detection and positioning of doorplate character outline：Pass through first The contour area of edge contour search function substantially searching character image, obtains the position coordinates matrix on profile boundary Contours obtains minimum enclosed rectangle；Secondly it to the boundary rectangle of acquisition, using unrestrained water filling algorithm, removes non-in background Character zone only retains doorplate character zone.

Step 3：For by the character picture after step 2, side is done to character picture first with edge detection Canny operators Edge detects, and obtains the substantially edge contour of character picture；Secondly, by the straightway met the requirements in Hough transformation detection figure, It should be noted that according to priori, character region is located substantially at 1/3 to 2/3 region of entire image, therefore first It determines 1/3 to 2/3 region that detection zone is entire image, Hough transformation is drawn out in figure successively in this region and is detected Every line segment, only retain length meet certain requirements and between two straight lines interval more than setting threshold value (threshold value herein Can adjust based on experience value) straightway；According to straight line position after detection, the starting point of every straight line is sequentially returned X, Y coordinate with terminal, according to starting and terminal point coordinate, it may be determined that the image-region between adjacent two lines separately protects it It deposits, is denoted as image dst (not provided for image storage class, can change according to actual conditions, similarly hereinafter), may finally obtain Thus the detection of character outline and positioning work are completed in the determining position of doorplate character.

Step 4：For the slope by character picture dst, first two straight line of calculating character up-and-down boundary after step 3, It is denoted as K1 and K2 respectively, for K1 and K2, seeks its mean value, is denoted as K；Secondly, using the center point coordinate in character picture region as rotation Turn center, character picture dst is rotated, angle corresponding to rotation angle, that is, slope K, the image after being rotated, note For image dst2, character picture is rotated horizontally.

It is finally obtained after being handled by above step complete and without image district where character existing for background interference factor Domain, and character meets common character order, the character cutting module after being lays the foundation.

5. existed according to described in claim 1 based on doorplate pressed characters identification device its feature under caffe and soft triggering In, to passing through the character picture after image pre-processing module in claim 4 for a bianry image dst2, the picture of every bit Value that there are two types of element values, i.e., 0 and 1, black and white is represented respectively, since character position is had determined positioned at the placed in the middle of image Position, external interference pixel have been removed, therefore character cutting module main purpose is to obtain cutting character picture, obtains list A character picture is simultaneously preserved.

The process flow of character cutting module is following steps：

Step 1：First, in claim 4 pass through image pre-processing module after character picture dst2, with a left side for image The starting point that upper angle is searched for as pixel value, for whole pictures, from top to bottom, progressive scan judges the pixel of every bit in image Value, main process flow are as follows：

(1) judges whether the pixel value is 0, if the pixel value is 0 (representing that the point is black), continues to scan on next Point pixel value；If the pixel value is 1 (representing that the point is white), judged (2).

(2) judges whether the ordinate is between 1/3 to the 2/3 of picture overall width, and ordinate is unsatisfactory for requiring Then return Rule of judgment 1, carry out the judgement of next point；If meeting the requirements, judged (3).

(3) on the basis of takes the ordinate, the point into each 30 pixel coverages of from left to right counts the pixel put in the range of this It is worth the pixel number for 1, judges whether the pixel number met the requirements is more than the 80% of the region all pixels point number, If being unsatisfactory for requiring, Rule of judgment 1 is returned, carries out the judgement of next point；The coordinate value of the point is returned if meeting the requirements, It is denoted as point P1 (X1, Y1).

Step 2：To, by the character picture dst2 after image pre-processing module, being made in claim 4 with the lower left corner of image For the starting point of pixel search, for whole pictures, progressive scan from bottom to up judges the pixel value of every bit in image, main Process flow with the process flow in step 1, thus obtained point coordinates value is denoted as P2 (X2, Y2).

Step 3：To, by the character picture dst2 after image pre-processing module, being made in claim 4 with the upper left corner of image For the starting point of pixel search, for whole pictures, the pixel value for judging every bit in image is scanned by column from left to right, it is main Process flow with the process flow in step 1, thus obtained point coordinates value is denoted as P3 (X3, Y3).

Step 4：To, by the character picture dst2 after image pre-processing module, being made in claim 4 with the upper left corner of image For the starting point of pixel search, for whole pictures, the pixel value for judging every bit in image is scanned by column from right to left, it is main Process flow with the process flow in step 3, thus obtained point coordinates value is denoted as P4 (X4, Y4).

Step 5：The specific of character is obtained according to the following formula in the coordinate value of four points obtained according to step 1 to step 4 Rectangle frame is confined in position, further reduces the location of character position：

Defined variable height and width, note：

Height=| Y2-Y1 |

Width=| X4-X3 |

A rectangle frame is confined on image dst2 in this way on artwork character picture and after binaryzation, it is believed that the rectangle frame is Accurate outer rim for character.

Step 6：To the character picture after determining accurate outer rim, need further to obtain the cutting result figure of single character, Main flow is as follows：

(1) understands that the model of pressed characters is big first, in accordance with priori for the character picture after determining accurate outer rim Small size is fixed, therefore first divides chinese character：Remember W3, H3 is the width and length of chinese character model, according to W3, H3 Length and width data, using W3 as standard on character picture, first determine first character, that is, chinese character substantially right margin；Again with The substantially right margin of chinese character is starting point, scans by column character picture from right to left, traverses the picture in each row character picture Element value, and if only if the pixel value be 1 (representing that the point be white) and on the basis of taking the abscissa, it is upward each 30 downward Point in pixel coverage, counts the pixel number that the pixel value put in the range of this is 1, and the pixel number met the requirements is more than The region all pixels point number 80% when, record the abscissa of current point, using the abscissa as axis, place straight line is the Chinese The accurate right margin of word character.Thus it can obtain the exact position of first character, i.e. chinese character.

(2) 6 characters that forms letter later and digital random, also according to W1, H1, W2, H2 (numeration word 0-9 The character boundary of template is W1, H1；Alphabetical A-Z template sizes be W2, H2) size width, first determine character substantially the right Boundary, then character picture is scanned by column from substantially right margin right-to-left, the pixel value in each row character picture is traversed, and then really Determine the accurate right margin of character, thus obtain the exact position of each character successively.

(3), by the picture that its size normalization is 28*28, carries out size and returns herein for the single character after Accurate Segmentation One purpose changed is that follow-up its input data size of used network model requirement is 28*28, size normalization will be passed through Single character later is denoted as cutting result figure.Cutting result figure is stored in corresponding file according to the content of corresponding character Under, character content is corresponded with folder name.

It should be noted that：First, cutting result figure is known as in follow-up identification module as training data source, training network Training cutting result figure；As test data source, the good and bad referred to as test cutting result of test network in follow-up identification module Figure；It is known as cutting result figure to be identified as character data collection to be identified in follow-up identification module.Secondly, according to actually adopting Collect picture number, using 60,000 character figures of the hardware collecting device module acquisition gained according to claims 1 as instruction Practice the data source of cutting result figure, it is another using data source of 10,000 collected character figures as test cutting result figure, it needs It is to be noted that 60,000 training sample sources and 10,000 test specimens origins herein are independent misaligned mutually.Finally, due to doorplate pressure Lettering symbol removes initial character and is fixed as except Chinese character, and the type and content of follow-up 6 characters are what is randomly generated, therefore cannot Ensure the cutting result figure uniform amount of number and letter distribution under variety classes, for the character cutting result under individual species If figure quantity it is very few, may be used basic image processing method (such as：Rotation, denoising, fuzzy, morphological operation etc. Deng) generation completion cutting result figure.

According to above-described six steps, the final single character picture that can obtain after character cutting as a result, and according to Character content is stored in correspondingly under corresponding file.

6. existed according to described in claim 1 based on doorplate pressed characters identification device its feature under caffe and soft triggering In again caffe study includes study module and identification module with identification module：Instruction of the study module according to obtained by claim 5 Practice the study that cutting result figure completes network model using the LeNet-5 hand-written scripts identification network processes in caffe, and according to The quality of test cutting result figure test current network model obtained by claim 5；Identification module is mainly cut according to be identified Point result figure is combined with gained network model, completes final identification work.

The specific process flow of study module is as follows：

Step 1：It before the device carries out character recognition, needs to learn network model, needs a large amount of input sample This, therefore manual sort is carried out for the training cutting result figure obtained by claim 5, corresponding character is saved in correspondence respectively Under the file of name, training sample set is formed.

Step 2：Manual sort is similarly carried out for the test cutting result figure obtained by claim 5, corresponding character is distinguished It is saved under the file of corresponding name, forms test sample collection.

Step 3：The picture that training sample is concentrated is tagged in sequence, upsets sequence, forms label file, and record Path where label file.The picture concentrated for test sample does identical operation, generates corresponding label file.

Step 4：Picture format is converted.Since the data format that input is required in LeNet-5 networks is ldb lmdb forms, Need the conversion of progress picture format.After conversion finishes, bat files are run, corresponding test set is obtained after being disposed With training set.

Step 5：Change the parameter under caffe parcels in LeNet-5 networks and path where file.

Step 6：Bat files are run, generate model file.Under caffe root paths, train_mnist.bat files are created simultaneously Content therein is modified, after running bat files, model file will be generated under respective path.

Step 7：Test network model is good and bad.In order in testing procedure six gained network model for other non-training samples How is the picture recognition effect of concentration, and network is tested using test sample, can test out the performance of the network.

After processing by above step one to step 7, you can obtain the model file after a training optimization Caffemodel will subsequently be identified character picture to be identified using it.

The specific process flow of identification module is as follows：Step 1：In device operational process, based on hard described in claim 1 Part collecting device module collected character picture to be identified, also pass through claim 2-5 described in flow obtain waiting to know Other cutting result figure.

Step 2：By the cutting result figure to be identified obtained by step 1 according to the step four in claim 6 learning module into Row processing, obtains the data set after format conversion, is denoted as character data collection to be identified.

Step 3：The model caffemodel files of gained in claim 6 learning module are called, by character data to be identified It is put into model and is identified, final output is with comparing the highest corresponding character of similarity probability value in model library, as currently The recognition result of character, then output is combined, thus as final doorplate pressed characters recognition result completes final word Symbol identification work.