CN108764257A - A kind of pointer instrument recognition methods of various visual angles - Google Patents
A kind of pointer instrument recognition methods of various visual angles Download PDFInfo
- Publication number
- CN108764257A CN108764257A CN201810500399.8A CN201810500399A CN108764257A CN 108764257 A CN108764257 A CN 108764257A CN 201810500399 A CN201810500399 A CN 201810500399A CN 108764257 A CN108764257 A CN 108764257A
- Authority
- CN
- China
- Prior art keywords
- instrument
- pointer
- carried out
- angle
- algorithms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The present invention relates to image identification technical fields.A kind of pointer instrument recognition methods of various visual angles, including：Image is acquired, and is uploaded to computer；Instrument zone location is carried out using SSD algorithms；Classification based training is carried out to instrument region using ResNet34 depth residual error neural networks, the preliminary correction of sample image is carried out according to classification results；The secondary positioning in instrument region is carried out to the image after correction using SSD algorithms；Regression training is carried out to the instrument region after secondary positioning using the network, identifies the pointer position on dial plate；Dial plate edge detection is carried out to the instrument region of positioning using HED edge detection algorithms；Stochastical sampling is carried out according to RANSAC algorithms, calculates the edge model of instrument dial plate；Gauge pointer is corrected using pantograph ratio, and calculates gauge pointer and originates the angle of needle；According to angle reference database table, the scale of instrument board is obtained.The pointer instrument that the present invention can shoot different angle is identified.
Description
Technical field
The present invention relates to the pointer instrument recognition methods of image identification technical field more particularly to a kind of various visual angles.
Background technology
Currently, most of reading of pointer instrument is carried out using artificial.And it manually reads and has the following disadvantages：1, instrument
It is placed on the place that people is difficult to enter, is read difficult；2, artificial to read it cannot be guaranteed that monitoring for a long time；3, operating personnel's vision is tired
Labor is also easy to produce reading error；4, artificial reading speed is slow.
Although replacing human eye to carry out the identification of pointer instrument using machine vision technique, can reduce to greatest extent artificial
The influence of factor can improve the speed of instrument reading, reduce the labor intensity of worker.But existing minority is regarded based on machine
The instrument read method of feel can obtain preferable reading effect in the ideal situation, but in practical applications, due to illumination
Uneven or pointer instrument can not easily lead to the collimation error, influence the precision of reading with the factors such as video camera keeping parallelism.
Number of patent application is 201710279811.3, entitled《A kind of pointer instrument identification side based on machine vision
Method》Chinese patent, disclose it is a kind of using machine vision technique replace artificial reading pointer type instrument registration read method,
By carrying out binary image processing, image outline extraction, scale profile cluster, the knowledge of scale fitting circle, pointer to Instrument image
Not, a series of image procossing and the calculating such as pointer refinement, the central coordinate of circle of solution calibration circle and scale expansion, reads instrument and shows
Number.
Number of patent application is 201710532391.5, entitled《Readings of pointer type meters based on NI Vision Builder for Automated Inspection is known
Not》Chinese patent, using the method for machine vision imaging, comprehensive realized using camera calibration and image recognition technology is surveyed
Amount.This method calculates pointer instrument dial plate in the position using the graduation mark of CCD camera detection pointer instrument dial plate
Under external parameter, replace traditional optical calibrating plate.By rotary indicator formula Instrument image to being parallel to pointer instrument table
Perspective projection distortion, the last reading for calculating multiple pointer instruments simultaneously are eliminated in the position of disk.In measurement process, do not require
Tested pointer instrument has specific position relationship with measuring system, relaxes and puts requirement to measuring system, is more advantageous to
Practical application.
It is to be directed to the instrument that dial plate footprint area is big, dial plate image clearly, shooting distance are close or front is shot mostly the above method
Table has preferable recognition effect, for the various visual angles of different angles such as there is side view, looking up, can not accurately read instrument
Indicate number.
Invention content
It is an object of the invention to overcome the shortcomings of existing pointer instrument recognition methods, a kind of finger of various visual angles is proposed
Pin type Meter recognition method.The pointer instrument that the present invention can shoot different angle be identified, and to side view, look up etc. no
Instrument with angle shot has stronger adaptability.
To achieve the goals above, the present invention uses following technical scheme：
A kind of pointer instrument recognition methods of various visual angles, includes the following steps：
Step 1：Image is acquired, Image Acquisition is carried out to pointer instrument using video camera, and be uploaded to computer；
Step 2：Instrument zone location is carried out using SSD algorithms；
Step 3：Classification based training is carried out to instrument region using ResNet34 depth residual error neural networks, is carried out according to classification results
The preliminary correction of sample image；The secondary positioning in instrument region is carried out to the image after correction using SSD algorithms；
Step 4：Regression training, identification are carried out to the instrument region after secondary positioning using ResNet34 depth residual error neural networks
Pointer position on dial plate；
Step 5：Dial plate edge detection is carried out to the instrument region of positioning using HED edge detection algorithms；According to RANSAC algorithms
Stochastical sampling is carried out, the edge model of instrument dial plate is calculated；
Step 6：Gauge pointer is corrected using pantograph ratio, and calculates gauge pointer and originates the angle of needle；
Step 7：According to the angle reference database table, the scale of instrument board is obtained.
Further, the step 2 includes：
Step 2.1：Sample data is pre-processed, pretreated sample data is obtained；
Step 2.2：SSD network models are built, in the infrastructure network of VGG16, by the 6th layer and the 7th layer of full articulamentum
It is converted to convolutional layer；Increase by 3 convolutional layers and an average pond layer；
Step 2.3：To every characteristic pattern after convolution, the coordinate after the recurrence of acquiescence frame is generated using 3 × 3 convolution and classification is general
Rate；Each the calculation formula of the size of the acquiescence frame is：
Wherein m is characterized map number, sminGive tacit consent to frame size, s for the bottommaxGive tacit consent to frame size for top；
Step 2.4：The pointer instrument region that definition has marked in advance is ground truth box, passes through ground truth
Box is trained SSD network models；Being accurately positioned for multi-angle pointer instrument is carried out using trained SSD networks；
Training process is as follows：
The acquiescence frame prior box and ground truth box actually chosen are matched according to IOU, IOU T1's
Prior box are positive sample, remaining is negative sample, the T1It is 0.7；The recurrence loss of prior box is carried out from high to low
Sequence select to return and loses highest M prior box as set D, the positive sample conduct set P after successful match, then just
Sample set is P-D ∩ P, and negative sample integrates as D-D ∩ P；The positive sample collection and negative sample concentrate the quantity of positive sample and negative sample
Than being 1:4, i.e. M are the 1/4 of prior box quantity；
Network parameter is adjusted by loss function, completes the positioning of pointer instrument；
The loss function is：
Wherein, c is class probability, and l is prediction block, and N is the prior box numbers to match with ground truth box；Such as
Fruit N=0, loss function 0；LconfFor Classification Loss part；Lloc(x, l, g) is prediction block l and g-th of ground truth
Part is lost in the recurrence of box；λ is the weight for returning loss, represents the contribution for returning loss to entire loss function, λ values
It is 0.5；
Step 2.5：Leave out repetition framework using NMS algorithms, chooses instrument region.
Further, the step 3 includes：
Step 3.1：Classification instruction is carried out to the instrument region that SSD algorithms detect using ResNet34 depth residual error neural networks
Practice, using below the middle finger needle of instrument region word or pattern and horizontal direction formed by angle a as criteria for classification, be trained；
Step 3.2：According to classification based training as a result, determining instrument region angle of inclination relative to horizontal direction, passes through matrix
Become scaling method to correct sample image artwork；
Step 3.3：Instrument zone location is carried out to the sample data figure after correction again using SSD algorithms.
Further, the step 4 is specially：Using ResNet34 depth residual error neural networks, to step 3, treated
Instrument disk area carries out regression training, is 0 degree starting needle to the left with horizontal direction, with (0,360] it is recurrence section, using prior
The instrument angle of mark returns pointer on instrument board and starting needle angle using ResNet34 depth residual error neural networks
Training obtains pointer using trained network and originates the angle of needle.
Further, the step 5 includes：
Step 5.1：Data after being detected using HED edge detection algorithms randomly select several as sampled point according to RANSAC algorithms
A sample point, the pixel separation between each sample point have to be larger than 20 pixels；
Step 5.2：Think that instrument plate edge is ellipse, is calculated according to the coordinate of the sample point and oval fundamental formular ellipse
Parameter in equation of a circle, the sample point for randomly selecting same number according to RANSAC algorithms again if no solution continue to count
It calculates, if there is solution, then judges whether the model solved is oval, and the model is given up if being unsatisfactory for condition, such as meets condition, then
Meet the sampled point number of following condition in statistic sampling point：
|dis(x,y,c_minus_x,c_minus_y)+dis(x,y,c_plus_x,c_plus_y)-2a|<5 (7)
Wherein, (x, y) is the coordinate of sampled point, and (c_minus_x, c_minus_y) and (c_plus_x, c_plus_y) is oval
Focal coordinates, dis (x, y, c_minus_x, c_minus_y) be sampled point (x, y) and elliptical focus (c_minus_x, c_
Minus_y distance), dis (x, y, c_plus_x, c_plus_y) be sampled point (x, y) and elliptical focus (c_plus_x,
C_plus_y distance), a are x in elliptic equation2Parameter；
Step 5.3：Step 5.1, step 5.2 are repeated, until having traversed all sampled points；
Step 5.4：It is optimal edge model to choose comprising the most elliptic equation of sampled point, according to optimal edge model
Elliptic equation obtains the corresponding elliptical center of elliptic equation and the coordinate of focus and the length of long axis.
Further, the step 6 includes：
Step 6.1：It is that Y-axis establishes coordinate system using horizontal direction as X-axis, vertical direction using elliptical center as coordinate axis center；
Step 6.2：Using elliptical center as the center of circle, positive round is drawn by radius of the half of elliptical long axis；
Step 6.3：It is 0 degree of starting needle to the left with horizontal direction, according to the pointer and starting needle using elliptical center as vertex
Angle, the picture pointer on ellipse；
Step 6.4：Pointer is corrected according to quadrant where pointer, eliminates the distortion of different angle shooting, realizes that instrument refers to
The correction of needle；It is as follows to correct formula：
Wherein, (x, y) be on ellipse pointer the coordinate with elliptical intersection point A online, where (x ', y ') is the pointer after correction
A point B on line, | a | and | b | be quadrant where pointer X-axis and Y-axis at a distance from ellipse intersection point to the center of circle；
Step 6.5：According to the coordinate of B points, the angle of the pointer after correction and starting needle is calculated.
Further, the step 7 is specially：The transfer standard for checking different type pointer meters in database will walk
Gauge pointer in rapid 6 is converted into corresponding scale with the angle of starting needle, obtains the registration of pointer meters.
Compared with prior art, the device have the advantages that：
Due to instrument function difference, cause its installation site different, the multiple angles shooting such as overlooks, looks up so existing
Instrument image.The shooting of various visual angles makes meter there are various perspective transforms in the picture, if do not corrected, the standard of instrument registration
Exactness can be greatly reduced.The present invention by being corrected to the Instrument image that different angle is shot, eliminate side view, look up etc. because
Influence, obtains good recognition effect, solves the identification problem of various visual angles pointer instrument caused by element.
Introduce deep learning algorithm, use SSD algorithm position indicator table sections first, make for different angle, different illumination,
The image of different resolution can effectively detect target area；Then ResNet34 depth residual errors neural network, HED are used
Edge detection algorithm and oval, positive round conversion method are combined, and are detected to instrument region, are eliminated what pointer meters were read
Visual angle error；Finally, instrument registration is identified using ResNet34 neural networks, realizes the instrument of different angle shooting
Automation is read.
Description of the drawings
Fig. 1 is a kind of basic flow chart of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention.
Fig. 2 is a kind of basic flow chart of the pointer instrument recognition methods of various visual angles of another embodiment of the present invention.
Fig. 3 is a kind of overall framework figure of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention.
Fig. 4 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
Zone location process schematic.
Fig. 5 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
Region correcting process schematic diagram.
Fig. 6 is a kind of optimal edge model signal of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
Figure.
Fig. 7 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
Pointer correcting process schematic diagram.
Fig. 8 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
One of test result schematic diagram.
Fig. 9 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
The two of test result schematic diagram.
Figure 10 is a kind of pointer instrument of the various visual angles of the pointer instrument recognition methods of various visual angles of the embodiment of the present invention
The three of table test result schematic diagram.
Specific implementation mode
Below in conjunction with the accompanying drawings with specific embodiment the present invention will be further explained explanation：
Embodiment one：
As shown in Figure 1, a kind of pointer instrument recognition methods of various visual angles of the present invention, includes the following steps：
Step S101：Image is acquired, Image Acquisition is carried out to pointer instrument using video camera, and be uploaded to computer；
Step S102：Instrument zone location is carried out using SSD algorithms；
Step S103：Classification based training is carried out to instrument region using ResNet34 depth residual error neural networks, according to classification
As a result the preliminary correction of sample image is carried out；The secondary positioning in instrument region is carried out to the image after correction using SSD algorithms；
Step S104：Recurrence instruction is carried out to the instrument region after secondary positioning using ResNet34 depth residual error neural networks
Practice, identifies the pointer position on dial plate；
Step S105：Dial plate edge detection is carried out to the instrument region of positioning using HED edge detection algorithms；According to
RANSAC algorithms carry out stochastical sampling, calculate the edge model of instrument dial plate；
Step S106：Gauge pointer is corrected using pantograph ratio, and calculates gauge pointer and originates the angle of needle；
Step S107：According to the angle reference database table, the scale of instrument board is obtained.
Embodiment two：
As shown in Figure 2 and Figure 3, the pointer instrument recognition methods of another various visual angles of the invention, includes the following steps：
Step 1：Image is acquired, Image Acquisition is carried out to pointer instrument using video camera, and be uploaded to computer.
Step 2：Pointer meters zone location.
Instrument zone location is carried out to collected sample data using SSD algorithms, Fig. 4 is instrument region location structure
Figure.
Specific implementation process is as follows：
1) pretreatment processing is carried out to sample data, obtains the sample data of 300 × 300 × 3 sizes.
2) SSD network models are built, in the infrastructure network of VGG16, the 6th layer and the 7th layer of full articulamentum is turned
It is melted into convolutional layer, characteristic pattern size is 38 × 38 and 19 × 19.Then 3 convolutional layers and an average pond layer are further added by, i.e.,
Pool layers of average, characteristic pattern size are respectively 10 × 10,5 × 5,3 × 3,1 × 1.
3) to every characteristic pattern after convolution, 4 recurrence of acquiescence frame (default boxes) are generated using 3 × 3 convolution
Coordinate afterwards and 4 class probabilities.
Each the calculation formula of acquiescence frame size is：
Wherein m is characterized map number, sminGive tacit consent to frame size, s for the bottommaxGive tacit consent to frame size for top.It is each silent
Recognize frame length-width ratio arRatio value be initially { 1,2,3,1/2,1/3 }, therefore, each width for giving tacit consent to frame isIt is a height ofThe acquiescence frame for being 1 for ratio, additionally adding a ratio isAcquiescence frame.Finally, every
Each of characteristic pattern point generates 6 acquiescence frames.Each acquiescence frame center is set asWherein, | fk| it is kth
A characteristic pattern size, i ∈ [1, k].
Due to smin、smaxValue directly affect the calculation amount of meter location algorithm, so referring to for collected various visual angles
Needle Instrument image carries out statistics calculating, determines smin=0.1, smax=0.25.And according to pointer meters in pointer meters image
The observation of shape counts, and when the ratio value for the length-width ratio for giving tacit consent to frame is revised as { 1,2,1/2 }, has just covered various visual angles
Meter further decreases the calculation amount of pointer meters location algorithm.S is demonstrated by a large amount of Experiment Training and testmin=
0.1, smax=0.25, when length-width ratio={ 1,2,1/2 }, for pointer meters recognizer under the premise of not reducing precision, the time is multiple
Miscellaneous degree is minimum.
4) SSD network models are trained by the image of gauge with pointer manually marked in advance, utilization is trained
Network carries out being accurately positioned for multi-angle pointer instrument.
First, positive and negative sample set is determined in conjunction with the image of gauge with pointer information manually marked.Pass through the instrument manually marked
Table section is known as ground truth box, and the ground truth box are by the true position data ground that correctly marks
Truth is formed, and prior box and ground truth box is matched according to IOU, the IOU is
JaccardOverlap, IOU>The prior box of T are exactly positive example, i.e. positive sample, and other is exactly negative
Example, i.e. negative sample.Since the value of T is most important to the recognition effect of meter integrality, instructed by many experiments
Practice, when determining T=0.85, the dial plate region that meter positioning result includes is more complete, does not influence further meter pointer and reads
Number identification.
Obviously, the quantity of negative sample collection is much larger than the quantity of positive sample collection, causes unbalanced between positive and negative sample set, makes
It is difficult to reach convergence when training.Therefore the corresponding recurrence losses of each prior box are ranked up, select highest num_
Sel prior box serial number is as set D.If the positive sample arrangement set after IOU matchings is P, last positive sample collection
For P-D ∩ P, negative sample integrates as D-D ∩ P, while the ratio of positive and negative samples is controlled by the quantity of num_sel.Due to positive and negative
The ratio of sample is particularly significant for being accurately positioned for meter, therefore the characteristics of according to the pointer instrument of various visual angles, through excessive
Secondary experimental contrast analysis determines that the ratio of positive and negative samples is 1:4, i.e. num_sel are the 1/4, Neng Gouqu of prior box quantity
Obtain faster convergence rate.
Then, network parameter is adjusted by loss function, completes the positioning of pointer instrument.Loss function includes two classes,
It is the recurrence loss loss (loc) and Classification Loss loss (conf) for giving tacit consent to frame respectively.Loss function is defined as：
Wherein, c is class probability, and l is prediction block, and N is prior box to match with ground truth box
Number；If N=0, loss function 0；LconfFor Classification Loss part, Classification Loss is measured using softmax loss functions；
Lloc(x, l, g) is prediction block l and part is lost in the recurrence of g-th of ground truth box；λ is the weight for returning loss, generation
Table returns contribution of the loss to entire loss function；Since λ value is most important to meter locating effect, regarded for a variety of
Pointer instrument under angle, by many experiments training and cross-beta, locating effect is optimal when determining λ=0.5.
Return loss part LlocIt is defined as follows：
Wherein l is prediction block, and g is ground truth, i.e. actual position, and p is the corresponding classifications of x, and d is acquiescence frame
(default bounding box)。
5) the instrument region for finally utilizing NMS algorithm picks best.
Step 3：Preliminary correction Instrument image.
Instrument angle of inclination is obtained using convolutional neural networks, by the preliminary correcting instrument table section of matrixing, using SSD
Reposition instrument region.It corrects as shown in figure 5, being as follows in instrument region：
1) ResNet34 depth residual error neural networks is used to carry out classification based training to the instrument region that SSD algorithms detect,
Using below the middle finger needle of instrument region word or pattern be trained as criteria for classification with angle a formed by horizontal direction；
2) according to classification based training as a result, determining instrument region angle of inclination relative to horizontal direction, is become by matrix
Scaling method corrects sample image artwork；
3) instrument zone location is carried out to the sample data figure after correction again using SSD algorithms.
Step 4：Pointer identifies.
Regression training is carried out to step 3 treated instrument board using ResNet34 depth residual error neural networks, with level
Direction is 0 degree of starting needle to the left, with (0,360] it is to return section.Using the instrument angle marked in advance, using ResNet34 depths
It spends residual error neural network and regression training is carried out to pointer on instrument board and starting needle angle, pointer is obtained using trained network
With the angle of starting needle.
Step 5：Instrument edge detection.
The instrument region chosen in step 3 is detected using HED edge detection algorithms, obtains pointer meters disk
Edge samples the edges detected HED using RANSAC algorithms, calculates oval instrument edge model, is obtained according to model
The length of the coordinate and long axis of elliptical center and focus.Pointer corrects specific method：
1) data after being detected using HED edge detection algorithms randomly select 5 samples as sampled point according to RANSAC algorithms
Point, the pixel separation between each sample point have to be larger than 20 pixels；
2) present invention approximately thinks that instrument plate edge is ellipse, according to the coordinate and ellipse of 5 sample points
Fundamental formular
ax2+bxy+cy2+ dx+ey+1=0 (5)
The parameter in elliptic equation is calculated, randomly selecting 5 sample points again according to RANSAC algorithms if no solution continues
It is calculated, if there is solution, then brings formula into
b2-4ac<0 (6)
Judge whether the model solved is oval, and the model is given up if being unsatisfactory for condition, such as meets condition, then statistics is adopted
Meet the sampled point number of following condition in sampling point
|dis(x,y,c_minus_x,c_minus_y)+dis(x,y,c_plus_x,c_plus_y)-2a|<5 (7)
Wherein, (x, y) is the coordinate of sampled point, and (c_minus_x, c_minus_y) and (c_plus_x, c_plus_y) is
Elliptical focal coordinates, dis (x, y, c_minus_x, c_minus_y) are sampled point (x, y) and elliptical focus (c_minus_
X, c_minus_y) distance, dis (x, y, c_plus_x, c_plus_y) be sampled point (x, y) and elliptical focus (c_
Plus_x, c_plus_y) distance, a be elliptic equation in x2Parameter；Above formula is using pixel as arithmetic unit.
1) and 2) 3) repeat in step 5, until having traversed all sampled points；
4) it is optimal edge model to choose comprising the most elliptic equation of sampled point, and the optimal edge model is as schemed
Shown in 6；
5) according to the equation of optimal edge model, the corresponding elliptical center of optimal edge model and focus are obtained
Coordinate and long axis length.
Step 6：Pointer is corrected.
According to the edge of gauge pointer and the angle and instrument dial plate of starting needle, pointer position is corrected using pantograph ratio, is obtained
It obtains pointer and originates the angle of needle.Pointer is corrected as shown in fig. 7, being as follows：
1) using the corresponding elliptical center of optimal edge model as coordinate axis center, using horizontal direction as X-axis, vertical direction
Coordinate system is drawn for Y-axis；
2) using the corresponding elliptical center of optimal edge model as the center of circle, with the optimal corresponding elliptical length of edge model
Axis half is that radius draws positive round；
3) it is 0 degree of starting needle to the left with horizontal direction using the corresponding elliptical center of optimal edge model as vertex, according to
The angle of pointer and starting needle, the picture pointer on ellipse in step 4；
4) pointer is corrected according to quadrant where pointer, eliminates the distortion of different angle shooting, realize gauge pointer
Correction.Its formula is as follows：
Wherein, (x, y) be on ellipse pointer the coordinate with elliptical intersection point A online, (x ', y ') be the pointer after correction
A point B on line, | a | and | b | be quadrant where pointer X-axis and Y-axis at a distance from ellipse intersection point to the center of circle.In Fig. 7
White pointer is the position of pointer in source images, and grey pointer is the position for correcting backpointer；
5) according to the coordinate of B points, the angle of the pointer after correction and starting needle is calculated.
Step 7：Convert scale.
The transfer standard for checking different type pointer meters in database, by the pointer after being corrected in step 6 and starting needle
Angle be converted into corresponding scale, obtain the registration of pointer meters.
Test results are shown in figure 8 for identification of the present embodiment method to pointer meters, and as shown in Figure 8, the present invention can be real
Automation reading now is carried out to the pointer instrument of different angle shooting.
Illustrated above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (7)
1. a kind of pointer instrument recognition methods of various visual angles, which is characterized in that include the following steps：
Step 1：Image is acquired, Image Acquisition is carried out to pointer instrument using video camera, and be uploaded to computer；
Step 2：Instrument zone location is carried out using SSD algorithms；
Step 3：Classification based training is carried out to instrument region using ResNet34 depth residual error neural networks, is carried out according to classification results
The preliminary correction of sample image；The secondary positioning in instrument region is carried out to the image after correction using SSD algorithms；
Step 4：Regression training, identification are carried out to the instrument region after secondary positioning using ResNet34 depth residual error neural networks
Pointer position on dial plate；
Step 5：Dial plate edge detection is carried out to the instrument region of positioning using HED edge detection algorithms；According to RANSAC algorithms
Stochastical sampling is carried out, the edge model of instrument dial plate is calculated；
Step 6：Gauge pointer is corrected using pantograph ratio, and calculates gauge pointer and originates the angle of needle；
Step 7：According to the angle reference database table, the scale of instrument board is obtained.
2. a kind of pointer instrument recognition methods of various visual angles according to claim 1, which is characterized in that the step 2
Including：
Step 2.1：Sample data is pre-processed, pretreated sample data is obtained；
Step 2.2：SSD network models are built, in the infrastructure network of VGG16, by the 6th layer and the 7th layer of full articulamentum
It is converted to convolutional layer；Increase by 3 convolutional layers and an average pond layer；
Step 2.3：To every characteristic pattern after convolution, the coordinate after the recurrence of acquiescence frame is generated using 3 × 3 convolution and classification is general
Rate；Each the calculation formula of the size of the acquiescence frame is：
Wherein m is characterized map number, sminGive tacit consent to frame size, s for the bottommaxGive tacit consent to frame size for top；
Step 2.4：The pointer instrument region that definition has marked in advance is ground truth box, passes through ground truth
Box is trained SSD network models；Being accurately positioned for multi-angle pointer instrument is carried out using trained SSD networks；
Training process is as follows：
The acquiescence frame prior box and ground truth box actually chosen are matched according to IOU, IOU T1Prior
Box is positive sample, remaining is negative sample, the T1It is 0.7；The recurrence loss of prior box is ranked up from high to low, is selected
It selects the highest M prior box of recurrence loss and is used as set D, the positive sample after successful match is used as set P, then positive sample collection
For P-D ∩ P, negative sample integrates as D-D ∩ P；It is 1 that the positive sample collection and negative sample, which concentrate the quantity ratio of positive sample and negative sample,:
4, i.e. M are the 1/4 of prior box quantity；
Network parameter is adjusted by loss function, completes the positioning of pointer instrument；
The loss function is：
Wherein, c is class probability, and l is prediction block, and N is the prior box numbers to match with ground truth box；Such as
Fruit N=0, loss function 0；LconfFor Classification Loss part；Lloc(x, l, g) is prediction block l and g-th of ground truth
Part is lost in the recurrence of box；λ is the weight for returning loss, represents the contribution for returning loss to entire loss function, λ values
It is 0.5；
Step 2.5：Leave out repetition framework using NMS algorithms, chooses instrument region.
3. a kind of pointer instrument recognition methods of various visual angles according to claim 1, which is characterized in that the step 3
Including：
Step 3.1：Classification instruction is carried out to the instrument region that SSD algorithms detect using ResNet34 depth residual error neural networks
Practice, using below the middle finger needle of instrument region word or pattern and horizontal direction formed by angle a as criteria for classification, be trained；
Step 3.2：According to classification based training as a result, determining instrument region angle of inclination relative to horizontal direction, passes through matrix
Become scaling method to correct sample image artwork；
Step 3.3：Instrument zone location is carried out to the sample data figure after correction again using SSD algorithms.
4. a kind of pointer instrument recognition methods of various visual angles according to claim 1, which is characterized in that the step 4
Specially：Regression training is carried out to step 3 treated instrument disk area using ResNet34 depth residual error neural networks, with water
Square to the left be 0 degree starting needle, with (0,360] be recurrence section, using the instrument angle marked in advance, using ResNet34
Depth residual error neural network carries out regression training to pointer on instrument board and starting needle angle, is referred to using trained network
The angle of needle and starting needle.
5. a kind of pointer instrument recognition methods of various visual angles according to claim 1, which is characterized in that the step 5
Including：
Step 5.1：Data after being detected using HED edge detection algorithms randomly select several as sampled point according to RANSAC algorithms
A sample point, the pixel separation between each sample point have to be larger than 20 pixels；
Step 5.2：Think that instrument plate edge is ellipse, is calculated according to the coordinate of the sample point and oval fundamental formular ellipse
Parameter in equation of a circle, the sample point for randomly selecting same number according to RANSAC algorithms again if no solution continue to count
It calculates, if there is solution, then judges whether the model solved is oval, and the model is given up if being unsatisfactory for condition, such as meets condition, then
Meet the sampled point number of following condition in statistic sampling point：
|dis(x,y,c_minus_x,c_minus_y)+dis(x,y,c_plus_x,c_plus_y)-2a|<5 (7)
Wherein, (x, y) is the coordinate of sampled point, and (c_minus_x, c_minus_y) and (c_plus_x, c_plus_y) is oval
Focal coordinates, dis (x, y, c_minus_x, c_minus_y) be sampled point (x, y) and elliptical focus (c_minus_x, c_
Minus_y distance), dis (x, y, c_plus_x, c_plus_y) be sampled point (x, y) and elliptical focus (c_plus_x,
C_plus_y distance), a are x in elliptic equation2Parameter；
Step 5.3：Step 5.1, step 5.2 are repeated, until having traversed all sampled points；
Step 5.4：It is optimal edge model to choose comprising the most elliptic equation of sampled point, according to optimal edge model
Elliptic equation obtains the corresponding elliptical center of elliptic equation and the coordinate of focus and the length of long axis.
6. a kind of pointer instrument recognition methods of various visual angles according to claim 4 or 5, which is characterized in that the step
Rapid 6 include：
Step 6.1：It is that Y-axis establishes coordinate system using horizontal direction as X-axis, vertical direction using elliptical center as coordinate axis center；
Step 6.2：Using elliptical center as the center of circle, positive round is drawn by radius of the half of elliptical long axis；
Step 6.3：It is 0 degree of starting needle to the left with horizontal direction, according to the pointer and starting needle using elliptical center as vertex
Angle, the picture pointer on ellipse；
Step 6.4：Pointer is corrected according to quadrant where pointer, eliminates the distortion of different angle shooting, realizes that instrument refers to
The correction of needle；It is as follows to correct formula：
Wherein, (x, y) be on ellipse pointer the coordinate with elliptical intersection point A online, where (x ', y ') is the pointer after correction
A point B on line, | a | and | b | be quadrant where pointer X-axis and Y-axis at a distance from ellipse intersection point to the center of circle；
Step 6.5：According to the coordinate of B points, the angle of the pointer after correction and starting needle is calculated.
7. a kind of pointer instrument recognition methods of various visual angles according to claim 1, which is characterized in that the step 7
Specially：The transfer standard for checking different type pointer meters in database, by the folder of gauge pointer and starting needle in step 6
Angle is converted into corresponding scale, obtains the registration of pointer meters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810500399.8A CN108764257B (en) | 2018-05-23 | 2018-05-23 | Multi-view pointer instrument identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810500399.8A CN108764257B (en) | 2018-05-23 | 2018-05-23 | Multi-view pointer instrument identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108764257A true CN108764257A (en) | 2018-11-06 |
| CN108764257B CN108764257B (en) | 2022-03-04 |
Family
ID=64004812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810500399.8A Active CN108764257B (en) | 2018-05-23 | 2018-05-23 | Multi-view pointer instrument identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764257B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109583324A (en) * | 2018-11-12 | 2019-04-05 | 武汉大学 | A kind of pointer meters reading automatic identifying method based on the more box detectors of single-point |
| CN109740483A (en) * | 2018-12-26 | 2019-05-10 | 南宁五加五科技有限公司 | A kind of rice growing season detection method based on deep-neural-network |
| CN109784257A (en) * | 2019-01-08 | 2019-05-21 | 北京全路通信信号研究设计院集团有限公司 | A kind of detection of transformer thermometer and recognition methods |
| CN109840497A (en) * | 2019-01-30 | 2019-06-04 | 华南理工大学 | A kind of pointer-type water meter reading detection method based on deep learning |
| CN109871754A (en) * | 2019-01-08 | 2019-06-11 | 深圳禾思众成科技有限公司 | A kind of instrument read method, equipment and computer readable storage medium |
| CN109948469A (en) * | 2019-03-01 | 2019-06-28 | 吉林大学 | The automatic detection recognition method of crusing robot instrument based on deep learning |
| CN110298339A (en) * | 2019-06-27 | 2019-10-01 | 北京史河科技有限公司 | A kind of instrument disk discrimination method, device and computer storage medium |
| CN110543878A (en) * | 2019-08-07 | 2019-12-06 | 华南理工大学 | pointer instrument reading identification method based on neural network |
| CN110659636A (en) * | 2019-09-20 | 2020-01-07 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
| CN111598094A (en) * | 2020-05-27 | 2020-08-28 | 深圳市铁越电气有限公司 | Deep learning-based angle regression meter reading identification method, device and system |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112560983A (en) * | 2020-12-24 | 2021-03-26 | 济南信通达电气科技有限公司 | Instrument identification method and device |
| CN112906681A (en) * | 2019-12-04 | 2021-06-04 | 杭州海康威视数字技术股份有限公司 | Meter reading method and device, electronic equipment and storage medium |
| CN113283419A (en) * | 2021-04-29 | 2021-08-20 | 国网浙江省电力有限公司湖州供电公司 | Convolutional neural network pointer instrument image reading identification method based on attention |
| CN113326787A (en) * | 2021-06-02 | 2021-08-31 | 武汉理工大学 | Automatic identification method, system and equipment for reading of pointer instrument |
| WO2021179485A1 (en) * | 2020-03-11 | 2021-09-16 | 平安科技（深圳）有限公司 | Image rectification processing method and apparatus, storage medium, and computer device |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113469302A (en) * | 2021-09-06 | 2021-10-01 | 南昌工学院 | Multi-circular target identification method and system for video image |
| CN113570631A (en) * | 2021-08-28 | 2021-10-29 | 西安安森智能仪器股份有限公司 | Intelligent pointer instrument identification method and equipment based on images |
| CN113743397A (en) * | 2021-11-08 | 2021-12-03 | 深圳市信润富联数字科技有限公司 | Data processing method and system for industrial instrument, terminal device and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080238818A1 (en) * | 2007-03-27 | 2008-10-02 | Denso Corporation | Display device |
| JP2009186401A (en) * | 2008-02-08 | 2009-08-20 | Calsonic Kansei Corp | Pointer meter for vehicle |
| CN104298994A (en) * | 2014-02-28 | 2015-01-21 | 郑州金惠计算机系统工程有限公司 | Pointer-type meter reading identification method and device |
| CN104657711A (en) * | 2015-02-09 | 2015-05-27 | 中国石油大学(华东) | Robust pointer instrument reading automatic identification method |
| CN106295665A (en) * | 2016-08-13 | 2017-01-04 | 国网福建省电力有限公司 | A kind of transformer station readings of pointer type meters automatic identifying method |
| CN107167169A (en) * | 2017-07-03 | 2017-09-15 | 吉林大学 | Readings of pointer type meters identification measuring method based on NI Vision Builder for Automated Inspection |
| CN107301412A (en) * | 2017-05-08 | 2017-10-27 | 浙江工业大学 | A kind of readings of pointer type meters method based on image procossing |
| CN107609557A (en) * | 2017-08-24 | 2018-01-19 | 华中科技大学 | A kind of readings of pointer type meters recognition methods |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
-
2018
- 2018-05-23 CN CN201810500399.8A patent/CN108764257B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080238818A1 (en) * | 2007-03-27 | 2008-10-02 | Denso Corporation | Display device |
| JP2009186401A (en) * | 2008-02-08 | 2009-08-20 | Calsonic Kansei Corp | Pointer meter for vehicle |
| CN104298994A (en) * | 2014-02-28 | 2015-01-21 | 郑州金惠计算机系统工程有限公司 | Pointer-type meter reading identification method and device |
| CN104657711A (en) * | 2015-02-09 | 2015-05-27 | 中国石油大学(华东) | Robust pointer instrument reading automatic identification method |
| CN106295665A (en) * | 2016-08-13 | 2017-01-04 | 国网福建省电力有限公司 | A kind of transformer station readings of pointer type meters automatic identifying method |
| CN107301412A (en) * | 2017-05-08 | 2017-10-27 | 浙江工业大学 | A kind of readings of pointer type meters method based on image procossing |
| CN107167169A (en) * | 2017-07-03 | 2017-09-15 | 吉林大学 | Readings of pointer type meters identification measuring method based on NI Vision Builder for Automated Inspection |
| CN107609557A (en) * | 2017-08-24 | 2018-01-19 | 华中科技大学 | A kind of readings of pointer type meters recognition methods |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
Non-Patent Citations (4)
| Title |
|---|
| WEI LIU等: "SSD：Single Shot MultiBox Detector", 《ARXIV》 * |
| ZHIJIA ZH.等: "Research on pointer location of multi-pointer meter", 《2009 IEEE INTERNATIONAL CONFERENCE ON AUTOMATION AND LOGISTICS》 * |
| 王欣 等: "智能变电站中指针式仪表图像自动识别方法", 《电气技术》 * |
| 邢浩强 等: "变电站指针式仪表检测与识别方法", 《仪器仪表学报》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109583324A (en) * | 2018-11-12 | 2019-04-05 | 武汉大学 | A kind of pointer meters reading automatic identifying method based on the more box detectors of single-point |
| CN109740483A (en) * | 2018-12-26 | 2019-05-10 | 南宁五加五科技有限公司 | A kind of rice growing season detection method based on deep-neural-network |
| CN109784257B (en) * | 2019-01-08 | 2021-10-12 | 北京全路通信信号研究设计院集团有限公司 | Transformer thermometer detection and identification method |
| CN109784257A (en) * | 2019-01-08 | 2019-05-21 | 北京全路通信信号研究设计院集团有限公司 | A kind of detection of transformer thermometer and recognition methods |
| CN109871754A (en) * | 2019-01-08 | 2019-06-11 | 深圳禾思众成科技有限公司 | A kind of instrument read method, equipment and computer readable storage medium |
| CN109840497A (en) * | 2019-01-30 | 2019-06-04 | 华南理工大学 | A kind of pointer-type water meter reading detection method based on deep learning |
| CN109948469A (en) * | 2019-03-01 | 2019-06-28 | 吉林大学 | The automatic detection recognition method of crusing robot instrument based on deep learning |
| CN109948469B (en) * | 2019-03-01 | 2022-11-29 | 吉林大学 | Automatic inspection robot instrument detection and identification method based on deep learning |
| CN110298339A (en) * | 2019-06-27 | 2019-10-01 | 北京史河科技有限公司 | A kind of instrument disk discrimination method, device and computer storage medium |
| CN110543878A (en) * | 2019-08-07 | 2019-12-06 | 华南理工大学 | pointer instrument reading identification method based on neural network |
| CN110659636B (en) * | 2019-09-20 | 2022-05-13 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
| CN110659636A (en) * | 2019-09-20 | 2020-01-07 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
| CN112906681A (en) * | 2019-12-04 | 2021-06-04 | 杭州海康威视数字技术股份有限公司 | Meter reading method and device, electronic equipment and storage medium |
| WO2021179485A1 (en) * | 2020-03-11 | 2021-09-16 | 平安科技（深圳）有限公司 | Image rectification processing method and apparatus, storage medium, and computer device |
| CN111598094A (en) * | 2020-05-27 | 2020-08-28 | 深圳市铁越电气有限公司 | Deep learning-based angle regression meter reading identification method, device and system |
| CN111598094B (en) * | 2020-05-27 | 2023-08-18 | 深圳市铁越电气有限公司 | Angle regression instrument reading identification method, equipment and system based on deep learning |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112560983A (en) * | 2020-12-24 | 2021-03-26 | 济南信通达电气科技有限公司 | Instrument identification method and device |
| CN112560983B (en) * | 2020-12-24 | 2023-09-08 | 济南信通达电气科技有限公司 | Instrument identification method and device |
| CN113283419A (en) * | 2021-04-29 | 2021-08-20 | 国网浙江省电力有限公司湖州供电公司 | Convolutional neural network pointer instrument image reading identification method based on attention |
| CN113283419B (en) * | 2021-04-29 | 2022-07-05 | 国网浙江省电力有限公司湖州供电公司 | Convolutional neural network pointer instrument image reading identification method based on attention |
| CN113326787A (en) * | 2021-06-02 | 2021-08-31 | 武汉理工大学 | Automatic identification method, system and equipment for reading of pointer instrument |
| CN113326787B (en) * | 2021-06-02 | 2023-09-01 | 武汉理工大学 | Automatic identification method, system and equipment for pointer instrument reading |
| CN113469178B (en) * | 2021-07-05 | 2024-03-01 | 安徽南瑞继远电网技术有限公司 | Power meter identification method based on deep learning |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113570631B (en) * | 2021-08-28 | 2024-04-26 | 西安安森智能仪器股份有限公司 | Image-based pointer instrument intelligent identification method and device |
| CN113570631A (en) * | 2021-08-28 | 2021-10-29 | 西安安森智能仪器股份有限公司 | Intelligent pointer instrument identification method and equipment based on images |
| CN113469302A (en) * | 2021-09-06 | 2021-10-01 | 南昌工学院 | Multi-circular target identification method and system for video image |
| CN113743397A (en) * | 2021-11-08 | 2021-12-03 | 深圳市信润富联数字科技有限公司 | Data processing method and system for industrial instrument, terminal device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108764257B (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764257A (en) | A kind of pointer instrument recognition methods of various visual angles | |
| US11551341B2 (en) | Method and device for automatically drawing structural cracks and precisely measuring widths thereof | |
| CN107229930B (en) | Intelligent identification method for numerical value of pointer instrument | |
| CN112906694B (en) | Reading correction system and method for transformer substation inclined pointer instrument image | |
| CN112818988B (en) | Automatic identification reading method and system for pointer instrument | |
| CN106778737B (en) | A kind of license plate antidote, device and a kind of video acquisition device | |
| CN111950330A (en) | Pointer instrument indicating number detection method based on target detection | |
| CN102704215B (en) | Automatic cutting method of embroidery cloth based on combination of DST file parsing and machine vision | |
| CN107657639A (en) | A kind of method and apparatus of quickly positioning target | |
| CN104700395A (en) | Method and system for detecting appearance crack of structure | |
| CN110261069B (en) | Detection method for optical lens | |
| CN113608378B (en) | Full-automatic defect detection method and system based on LCD (liquid crystal display) process | |
| CN103776482B (en) | The image detecting method of the scale of pointer instrument without scale line | |
| CN111738229B (en) | Automatic reading method for scale of pointer dial | |
| CN111724354B (en) | Image processing-based method for measuring wheat ear length and wheat ear number of multiple wheat plants | |
| CN113688817A (en) | Instrument identification method and system for automatic inspection | |
| CN113884011A (en) | Non-contact concrete surface crack measuring equipment and method | |
| CN105391998B (en) | Automatic detection method and apparatus for resolution of low-light night vision device | |
| CN110363706A (en) | A kind of large area bridge floor image split-joint method | |
| CN116091488B (en) | Displacement testing method and displacement testing system for engine swing test | |
| CN107883825A (en) | A kind of intelligent visual detection method for the slide measure zero graduation line that aligns | |
| CN112001964A (en) | Flood evolution process inundation range measuring method based on deep learning | |
| CN114092728A (en) | Pointer instrument intelligent identification method and system based on deep learning | |
| CN111189826B (en) | Intelligent scoring experimental equipment and method for measuring pH value of solution to be measured by pH test paper | |
| CN115682957A (en) | Concrete crack detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |