CN110543878A - pointer instrument reading identification method based on neural network - Google Patents
pointer instrument reading identification method based on neural network Download PDFInfo
- Publication number
- CN110543878A CN110543878A CN201910724076.1A CN201910724076A CN110543878A CN 110543878 A CN110543878 A CN 110543878A CN 201910724076 A CN201910724076 A CN 201910724076A CN 110543878 A CN110543878 A CN 110543878A
- Authority
- CN
- China
- Prior art keywords
- layer
- convolution
- loss
- dial
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a pointer instrument reading identification method based on a neural network, which comprises the following steps: 1) constructing and preprocessing a dial positioning data set; 2) designing a matched neural network model aiming at the characteristics of the identified object and the application scene; 3) training the loading training parameters of the neural network model to obtain a dial plate positioning model; 4) inputting an image to be recognized into a trained dial plate positioning model to obtain the position and the category of a dial plate in the image, outputting and constructing a dial plate information extraction data set and preprocessing the dial plate information extraction data set by the dial plate positioning model, loading training parameters on a neural network model and training on the dial plate information extraction data set, and obtaining a dial plate information extraction model after training is finished; 5) outputting and inputting the dial plate positioning model into the trained dial plate information extraction model to obtain dial plate range information; 6) extracting the position information of the pointer and the dial center from the dial positioning model output; 7) the meter reading is calculated. The invention can obtain high-precision meter reading on the premise of ensuring real-time performance.
Description
Technical Field
The invention relates to the technical field of pattern recognition and artificial intelligence, in particular to a pointer instrument reading recognition method based on a neural network.
background
the pointer instrument is widely applied to various fields such as industrial production, social life, environmental monitoring and the like, and provides a lot of convenience for production work. At present, manual reading is a widely adopted reading method, but the manual reading has the defects of low efficiency, large error, poor stability and the like, and especially in places which are not suitable for people to enter, such as the high-voltage environment of a transformer substation, the manual reading loses feasibility. In recent years, with the rapid development of machine vision and artificial intelligence technologies, an automatic meter reading identification technology based on image equipment becomes possible, the application scene limitation of an artificial reading method can be effectively avoided by using the automatic identification technology, and the method has the advantages of improving efficiency, eliminating errors and the like. Therefore, the research on the pointer instrument reading automatic identification technology has very important significance.
at present, methods for automatically identifying the reading of a pointer instrument are many, and the methods can be mainly divided into a method based on traditional image processing and a method based on deep learning. The traditional image processing method mainly comprises a subtraction method, a Hough transform detection method, a template matching method and the like, but the methods have some defects, for example, the subtraction method not only needs to obtain a zero-scale image of an instrument to be identified in advance, but also needs a background area to be identified to be unchanged, the Hough transform detection method only can detect a dial plate and a pointer with relatively standard shapes, a template image with a single size needs to be obtained in template matching, relatively high contrast is needed between the template image and the background, the robustness of the traditional image processing methods is easily influenced by factors such as illumination, shooting angle and imaging quality, and the application scene is limited. The deep learning method based on the deep learning can be used for positioning and identifying various types of instruments in different scenes after a large number of data sets are trained, has high robustness, low requirements for application scenes and strong popularization, but the current mainstream deep learning method needs a large amount of data sets and time for training, and cannot ensure real-time performance under limited hardware conditions due to a complex structure. Therefore, structural optimization needs to be performed on the basis of the existing deep learning technology, the algorithm training cost is reduced, and the algorithm instantaneity is improved.
In combination with the above discussion, the pointer instrument reading identification method with real-time performance and high precision has higher practical application value.
Disclosure of Invention
the invention aims to overcome the defects of the prior art and provides a pointer instrument reading identification method based on a neural network, which mainly utilizes a deep learning technology design to realize dial plate positioning, classification and range information extraction functions, then utilizes an image processing technology to obtain pointer and circle center position information, and utilizes an angle method to obtain high-precision instrument reading on the premise of ensuring real-time performance.
in order to achieve the purpose, the technical scheme provided by the invention is as follows: a pointer instrument reading identification method based on a neural network comprises the following steps:
1) constructing and preprocessing a dial positioning data set;
2) designing a matched neural network model aiming at the characteristics of the identified object and the application scene;
3) training the designed neural network model loading training parameters, performing online data enhancement in the training process, and obtaining a dial plate positioning model after the training is finished;
4) Inputting an image to be recognized into a trained dial plate positioning model to obtain the position and the category of a dial plate in the image, outputting and constructing a dial plate information extraction data set by the dial plate positioning model and preprocessing, loading training parameters on the neural network model designed in the step 2) and training on the dial plate information extraction data set, using online data enhancement in the training process, and obtaining a dial plate information extraction model after the training is finished;
5) Outputting and inputting the dial plate positioning model into the trained dial plate information extraction model to obtain dial plate range information;
6) Obtaining the position of a pointer and the central position of the dial by the output of the dial positioning model and the output of the dial information extraction model through an image processing technology;
7) Obtaining a maximum range position and a minimum range position from the output of the dial information extraction model so as to obtain a total range angle, obtaining a reading angle from the pointer position and the minimum range position, and obtaining the reading of the instrument by using the same proportional relation, namely an angle method; wherein, the formula of the angle method is as follows:
where num is the pointer meter reading, angle1 is the angle between the maximum range and the minimum range, angle2 is the angle between the minimum range and the pointer position, max is the maximum range reading, and min is the minimum range reading.
in the step 1), pointer instrument image data under different scenes are collected through image collection equipment, an original data set is constructed, then interference data influencing neural network training and recognition are removed, the interference data include fuzzy data, extreme angle data and dial plate missing data, and other data are marked, wherein the marking content is the position and the category of the dial plate of the instrument.
In step 2), a matched neural network is constructed by combining the characteristics of the recognition object and the application scene, and the method comprises the following steps:
2.1) constructing a feature extraction network
the method comprises the following steps of constructing a feature extraction network according to real-time and high-precision requirements, wherein the feature extraction network mainly comprises a plurality of combined convolution modules and has the structure as follows:
The first layer is a combined convolution module A which consists of a zero filling layer, a convolution layer, a batch normalization layer and an activation layer;
The second layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The third layer is a combined convolution module C which consists of a zero filling layer, a deep convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The fourth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The fifth layer is a combined convolution module C which consists of a zero filling layer, a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The sixth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The seventh layer is a combined convolution module C which consists of a zero filling layer, a deep convolution layer, two batch normalization layers, two activation layers and a convolution layer;
the eighth layer is a combined convolution module D which consists of five combined convolution modules B;
The ninth layer is a combined convolution module C which consists of a zero filling layer, a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The tenth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
2.2) building a prediction network
according to the output of different layers of the feature extraction network, a prediction network for outputting and predicting targets with different sizes is constructed, and the method comprises the following steps:
a. large-size target prediction network
the input is the tenth layer output of the feature extraction network, the large-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the structure of the large-size target prediction network is as follows:
The first layer is a combined convolution module D which consists of five combined convolution modules B;
the second layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The third layer is a convolution layer;
b. medium size target prediction network
The input is the eighth layer output of the feature extraction network and the first layer output of the large-size target prediction network, the medium-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the structure of the medium-size target prediction network is as follows:
The first layer is an input fusion module which consists of a combined convolution module B, an up-sampling layer and a tensor splicing layer;
the second layer is a combined convolution module D which consists of five combined convolution modules B;
The third layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The fourth layer is a convolution layer;
c. Small size target prediction network
the input is the sixth layer output of the feature extraction network and the second layer output of the medium-size target prediction network, the small-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the small-size target prediction network has the structure that:
The first layer is an input fusion module which consists of a combined convolution module B, an up-sampling layer and a tensor splicing layer;
the second layer is a combined convolution module D which consists of five combined convolution modules B;
The third layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The fourth layer is a convolution layer;
Finally, the output of the large-size target prediction network, the medium-size target prediction network and the small-size target prediction network passes through a non-maximum value inhibition layer to obtain the positions and the types of the predicted targets;
2.3) setting the loss function
setting a loss function as a summation mean of a central coordinate loss function, a width and height loss function, confidence coefficient loss and a category loss function, wherein the loss function formula is as follows:
Loss＝(Loss+Loss+Loss+Loss)/numf
wherein, Loss represents total Loss, Lossxy represents central coordinate Loss, Losswh represents width and height Loss, lossconfident represents confidence Loss, Losscls represents category Loss, and numf represents floating point number of input total number; the respective loss functions are formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
Loss＝0.5*mark*(2-w*h)*(whtrue-wh)
Loss＝mark*Loss(mark,c)+(1-mark)*Loss
(mark,c)*mark
Loss＝mark*Loss(cls,cls)
The center coordinate loss function is formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
wherein markobject represents whether the anchor frame has a mark bit of an object, w represents the width of the anchor frame, h represents the height of the anchor frame, Losslog represents binary cross entropy loss, xytrue represents a real center coordinate value, xypredict represents a predicted center coordinate value, whtrutrutrure represents a real width and height value, whredict represents a predicted width and height value, cpredict represents a confidence value of the prediction frame, markignore represents a mark bit of the anchor frame with an IOU smaller than a threshold value, clstrie represents a real class, and clsprredict represents a prediction class.
in step 3), training the designed neural network model, comprising the following steps:
3.1) setting training parameters
setting a training optimizer as Adam, an initial learning rate of 0.001, an iteration number of 500, a batch size of 8 and K means clustering on all labels to generate initial prior frames of (38, 29), (65, 52), (94, 87), (142, 134), (195, 69), (216, 206), (337, 320), (397, 145), (638, 569);
3.2) Online data enhancement
The data enhancement is carried out on the input image, the data set is expanded, and the main method of the data enhancement is as follows:
a. random mirror inversion
carrying out random mirror image overturning on an input image;
b. random additive noise
Adding a continuous single noise mask to the input image;
c. randomly adjusting contrast
Modifying hue and saturation to realize contrast conversion;
3.3) setting training completion flag
and setting verification set interval detection training accuracy, marking a training completion mark to meet the requirements of the maximum iteration number and the accuracy, and storing the model structure and parameters after the training is completed.
In step 4), integrating the output images obtained by the dial plate positioning model into a dial plate information extraction training set, performing labeling work, wherein the labeled content is range information, and training the neural network model designed in step 2) on the dial plate information extraction data set, wherein the method comprises the following steps:
4.1) setting training parameters
Setting a training optimizer as Adam, an initial learning rate of 0.001, an iteration number of 600, a batch size of 8 and K means clustering on all labels to generate initial prior frames of (8, 12), (10, 16), (11, 20), (12, 12), (14, 16), (15, 26), (21, 19), (24, 34), (45, 55);
4.2) Online data enhancement
the data enhancement is carried out on the input image, the data set is expanded, and the main method of the data enhancement is as follows:
a. Random mirror inversion
Carrying out random mirror image overturning on an input image;
b. random addition of rectangular noise
Information loss in a plurality of rectangular areas with selectable area sizes and random positions in an input image is converted, information loss of all channels generates black rectangular blocks, and information loss of partial channels generates color noise;
c. Randomly adjusting contrast
modifying hue and saturation to realize contrast conversion;
4.3) setting training completion flag
and setting verification set interval detection training accuracy, marking a training completion mark to meet the requirements of the maximum iteration number and the accuracy, and storing the model structure and parameters after the training is completed.
In the step 5), a dial area image is obtained by the output of the dial positioning model, and the dial area image is input into the dial information extraction model to obtain dial range information.
in step 6), obtaining the pointer position and the dial center position by the dial positioning model output and the dial information extraction model output through an image processing technology, and the method comprises the following steps:
6.1) Dial correction
obtaining a transformation matrix by using dial plate central range position information and central range real position information which are obtained by the output of the dial plate information extraction model, and correcting a dial plate image through affine transformation;
6.2) pointer location segmentation
Obtaining a binary dial image by applying an OTSU (optical transmission unit) adaptive threshold segmentation method, searching a maximum contour meeting the length-width ratio and the area of a specified range on the binary image to obtain a pointer initial position, and obtaining the pointer position through Hough line detection and least square fitting;
6.3) Dial centering
And obtaining the center of the dial to be selected by using Hough circle detection, calculating the Euclidean distance between each center and the pointer, and obtaining the center position of the dial according to the minimum distance principle.
compared with the prior art, the invention has the following advantages and beneficial effects:
1. The dial positioning and the dial information extraction are completed by using the neural network, the positions and the range information of the pointer instruments with different backgrounds and different types can be accurately identified in a complex environment, and the accuracy is higher than that of a method based on the traditional image processing technology.
2. the designed neural network adopts deep separable convolution to enable the parameter quantity to be small, the network structure is simple, a high frame rate can be obtained on the premise of guaranteeing high accuracy so as to guarantee real-time performance, and the requirement on hardware is low.
3. The target prediction network can predict targets with different sizes, comprehensively select a prediction frame with the most suitable size, accurately detect instruments with different sizes, and is free from the limitation of collection angle and distance.
4. The designed loss function comprises the confidence loss of the background class, and the influence of unbalance of positive and negative samples on training can be effectively eliminated.
drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2a is a schematic diagram of the combined convolution module a.
fig. 2B is a schematic diagram of the combined convolution module B.
fig. 2C is a schematic diagram of the combined convolution module C.
fig. 2D is a schematic diagram of the combined convolution module D.
FIG. 3 is a schematic diagram of an input fusion module.
Fig. 4 is a schematic view of an angle reading identification method.
where num is the pointer meter reading, angle1 is the angle between the maximum range and the minimum range, angle2 is the angle between the pointer and the minimum range, max is the maximum range reading, and min is the minimum range reading.
Detailed Description
The present invention will be further described with reference to the following specific examples.
As shown in fig. 1, the method for identifying a reading of a pointer instrument based on a neural network provided in this embodiment includes the following steps:
1) and collecting pointer instrument images shot in different actual scenes to construct an original data set. Interference data which affect neural network training and recognition due to the existence of fuzziness, extreme angles, dial plate missing and the like are removed, dial plate positions and categories in the rest data are marked by using an open source marking tool labelImg, and a dial plate positioning training set is constructed.
2) Designing a neural network meeting actual requirements according to the specific application scene and the characteristics of an identification object, wherein the following activation layers are all Leaky Relu activation functions if not additionally stated, and the method comprises the following steps:
2.1) constructing a feature extraction network
And constructing a feature extraction network according to the requirements of real-time performance and high precision. The feature extraction network is mainly composed of a plurality of combined convolution modules.
the feature extraction network structure is as follows:
The input images are 416x416x 3.
the first layer is the combined convolution module a, as shown in fig. 2 a. The module first passes through the zero-padding layer with an output of 418 x 3. Then the convolution kernel is (3, 3), the step length is 2, the filter number is 32, and the output is 208 multiplied by 32.
The second layer is the combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 208 × 208 × 32. And after the convolution, batch normalization and activation layers, the convolution kernel is (1, 1), the step size is 1, the number of filters is 64, the input and output sizes are consistent by using filling, and the output is 208 multiplied by 64.
the third layer is a combined convolution module C, as shown in fig. 2C. The module first passes through the zero-padding layer and the output is 210 x 64. And then the obtained product is subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 2, and the output is 104 multiplied by 64. And finally, after convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 128, the input and output sizes are consistent by using filling, and the output is 104 multiplied by 128.
the fourth layer is the combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, the input and output sizes are consistent by using padding, and the output is 104 × 104 × 128. And then the filter passes through a convolution layer, a batch normalization layer and an activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 128, the input and output sizes are consistent by using filling, and the output is 104 multiplied by 128.
the fifth layer is a combined convolution module C, as shown in fig. 2C. The module first passes through the zero-padding layer with an output of 106 x 128. And then the obtained product is subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 2, and the output is 52 multiplied by 128. And finally, after convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 52 multiplied by 256.
the sixth layer is the combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 52 × 52 × 256. And then the filter passes through a convolution layer, a batch normalization layer and an activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 52 multiplied by 256.
the seventh layer is a combined convolution module C, as shown in fig. 2C. The module first passes through the zero-padding layer and the output is 54 x 256. And then the obtained product is subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 2, and the output is 26 multiplied by 256. And finally, after convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 512.
the eighth layer is the combined convolution module D, as shown in fig. 2D. The modules pass through five combined convolution modules B in sequence as shown in fig. 2B. In each combined convolution module B, the input first passes through a deep convolution, a batch normalization layer and an activation layer, the convolution kernel is (3, 3), the step size is 1, the input and output sizes are consistent by using padding, and the output size is 26 multiplied by 512. And (3) performing convolution, batch normalization and activation, wherein the convolution kernel is (1, 1), the step size is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 512. After sequentially passing through the same combined convolution module B, the output is 26 × 26 × 512.
The ninth layer is a combined convolution module C, as shown in fig. 2C. The module first passes through the zero-padding layer and the output is 28 x 512. And then the obtained product is subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step length is 2, and the output is 13 multiplied by 512. And finally, performing convolution, batch normalization and activation layers, wherein the convolution kernel is (1, 1), the step length is 1, the number of filters is 1024, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 1024.
the tenth layer is the combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, the input and output sizes are consistent by using padding, and the output is 13 × 13 × 1024. And then the convolution, batch normalization and activation layers are carried out, the convolution kernel is (1, 1), the step length is 1, the number of filters is 1024, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 1024.
2.2) building a prediction network
And constructing a prediction network for outputting and predicting the targets with different sizes according to the output of different layers of the feature extraction network.
a. Large-size target prediction network
the input is the tenth layer output of the feature extraction network, and the large-size target prediction network mainly comprises a plurality of combined convolution modules, convolution layers and other neural network modules.
the input image is 13 × 13 × 1024.
the large-size target prediction network structure is as follows:
The first layer is the combined convolution module D, as shown in fig. 2D. The modules pass through five combined convolution modules B in sequence as shown in fig. 2B. In the first combined convolution module B, the input first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 13 × 13 × 1024. And performing convolution, batch normalization and activation layers, wherein the convolution kernel is (1, 1), the step length is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 512. In the second combined convolution module B, the input is first subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 13 × 13 × 512. And then the convolution, batch normalization and activation layers are carried out, the convolution kernel is (1, 1), the step length is 1, the number of filters is 1024, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 1024. After the two different parameters of the combined convolution module B are alternately input, the output is 13 multiplied by 512.
The second layer is the combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, the input and output sizes are consistent by using padding, and the output is 13 × 13 × 512. And then the convolution, batch normalization and activation layers are carried out, the convolution kernel is (1, 1), the step length is 1, the number of filters is 1024, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 1024.
The third layer is a convolutional layer. The convolution kernel is (1, 1), the step size is 1, the number of filters is 255, and the output is 13 × 13 × 255.
b. Medium size target prediction network
the input is the eighth layer output of the feature extraction network and the first layer output of the large-size target prediction network, and the medium-size target prediction network mainly comprises a plurality of combined convolution modules, convolution layers and other neural network modules.
The input images are 26 × 26 × 512 and 13 × 13 × 512.
The medium-sized target prediction network structure is as follows:
The first layer is the input fusion module, as shown in FIG. 3. The input 13 x 512 first goes through the combined convolution module B where first the deep convolution, batch normalization layer and activation layer are passed, the convolution kernel is (1, 1) and the step size is 1, padding is used to make the input and output sizes consistent and the output is 13 x 512. And performing convolution, batch normalization and activation layers, wherein the convolution kernel is (1, 1), the step length is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 13 multiplied by 512. And then passes through an up-sampling layer, the sampling factor is 2, and the output is 26 multiplied by 512. Finally, the output and input are 26 × 26 × 512 through a tensor splicing layer, and the output is 26 × 26 × 1024.
the second layer is the combined convolution module D, as shown in fig. 2D. The modules pass through five combined convolution modules B in sequence as shown in fig. 2B. In the first combined convolution module B, the input first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 26 × 26 × 1024. And then the data is subjected to convolution, batch normalization and activation layers, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 256. In the second combined convolution module B, the input is first subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 26 × 26 × 256. And (3) performing convolution, batch normalization and activation, wherein the convolution kernel is (1, 1), the step size is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 512. After the two different parameters of the combined convolution module B are alternately input, the output is 26 multiplied by 256.
the third layer is a combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, the input and output sizes are consistent by using padding, and the output is 26 × 26 × 256. And (3) performing convolution, batch normalization and activation, wherein the convolution kernel is (1, 1), the step size is 1, the number of filters is 512, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 512.
The fourth layer is a convolutional layer. The convolution kernel is (1, 1), the step size is 1, the number of filters is 255, and the output is 26 × 26 × 255.
c. small size target prediction network
The input is the sixth layer output of the feature extraction network and the second layer output of the medium-size target prediction network, and the small-size target prediction network mainly comprises a plurality of combined convolution modules, convolution layers and other neural network modules.
the input images are 52 × 52 × 256 and 26 × 26 × 256.
the small-size target prediction network structure is as follows:
The first layer is the input fusion module, as shown in FIG. 3. The input 26 × 26 × 256 first passes through the combined convolution module B, where the deep convolution, batch normalization layer and activation layer are first passed, the convolution kernel is (1, 1), the step size is 1, padding is used to make the input and output size uniform, and the output is 26 × 26 × 256. And then the data is subjected to convolution, batch normalization and activation layers, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 26 multiplied by 256. And the sampling factor is 2 after passing through an up-sampling layer, and the output is 52 multiplied by 256. Finally, the output and input 52 × 52 × 256 go through a tensor concatenation layer, and the output is 52 × 52 × 512.
The second layer is the combined convolution module D, as shown in fig. 2D. The modules pass through five combined convolution modules B in sequence as shown in fig. 2B. In the first combined convolution module B, the input first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (1, 1), the step size is 1, padding is used to make the input and output size consistent, and the output is 52X 512. And then the filter passes through a convolution layer, a batch normalization layer and an activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 128, the input and output sizes are consistent by using filling, and the output is 52 multiplied by 128. In the second combined convolution module B, the input is first subjected to deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 52X 128. And then the filter passes through a convolution layer, a batch normalization layer and an activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 52 multiplied by 256. After the two different parameters of the combined convolution module B are alternately input, the output is 52 multiplied by 128.
The third layer is a combined convolution module B, as shown in fig. 2B. The module first goes through deep convolution, batch normalization layer and activation layer, the convolution kernel is (3, 3), the step size is 1, padding is used to make the input and output sizes consistent, and the output is 52 × 52 × 128. And then the filter passes through a convolution layer, a batch normalization layer and an activation layer, the convolution kernel is (1, 1), the step size is 1, the number of filters is 256, the input and output sizes are consistent by using filling, and the output is 52 multiplied by 256.
The fourth layer is a convolutional layer. The convolution kernel is (1, 1), the step size is 1, the number of filters is 255, and the output is 52 × 52 × 255.
and finally, obtaining the predicted target position and the predicted target category through a non-maximum value suppression layer by using the output 13 × 13 × 255 of the large-size target prediction network, the output 26 × 26 × 255 of the medium-size target prediction network and the output 52 × 52 × 255 of the small-size target prediction network.
2.3) setting the loss function
and setting the loss function as a summation mean of a central coordinate loss function, a width and height loss function, a confidence coefficient loss and a category loss function. The loss function is formulated as follows:
Loss＝(Loss+Loss+Loss+Loss)/numf
Wherein Loss represents total Loss, Lossxy represents center coordinate Loss, Losswh represents width and height Loss, lossconfident represents confidence Loss, Losscls represents category Loss, and numf represents floating point number of total input. The respective loss functions are formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
Loss＝0.5*mark*(2-w*h)*(wh-wh)
Loss＝mark*Loss(mark,c)+(1-mark)*Loss
(mark,c)*mark
Loss＝mark*Loss(cls,cls)
the center coordinate loss function is formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
Wherein markobject represents whether the anchor frame has a mark bit of an object, w represents the width of the anchor frame, h represents the height of the anchor frame, Losslog represents binary cross entropy loss, xytrue represents a real center coordinate value, xypredict represents a predicted center coordinate value, whtrutruru represents a real width and height value, whredict represents a predicted width and height value, cpredict represents a confidence value of the prediction frame, markignore represents a mark bit of the anchor frame with an IOU smaller than a threshold value, clspredict represents a prediction category.
3) training the designed neural network model, comprising the steps of:
3.1) setting training parameters
Setting Adam as a training optimizer, 0.001 as an initial learning rate, 500 as an iteration number, 8 as a batch size, and K means clustering on all labels generates initial prior frames (38, 29), (65, 52), (94, 87), (142, 134), (195, 69), (216, 206), (337, 320), (397, 145), (638, 569).
3.2) Online data enhancement
The data enhancement is carried out on the input image, the data set is expanded, and the data enhancement method comprises the following steps:
a. Random mirror inversion
and carrying out random mirror image inversion on the input image.
b. Random additive noise
A continuous single noise mask is added to the input image.
c. Randomly adjusting contrast
modifying hue and saturation effects contrast conversion.
3.3) setting training completion flag
Setting verification set interval detection training accuracy, setting a training completion flag to reach a maximum iteration number of 500, enabling the accuracy to meet 99% requirements, and storing model structures and parameters after training is completed.
4) Integrating output images obtained through a dial plate positioning model into a dial plate information extraction training set, eliminating interference data which have the influences on neural network training and recognition such as fuzziness, extreme angles, dial plate deletion and the like, marking positions and categories of dimensions in other data by using an open source marking tool, namely label img, constructing the dial plate information extraction training set, and training the neural network model designed in the step 2) on the dial plate information extraction data set, wherein the method comprises the following steps:
4.1) setting training parameters
setting a training optimizer as Adam, an initial learning rate of 0.001, an iteration number of 600, a batch size of 8 and K means clustering on all labels to generate initial prior frames of (8, 12), (10, 16), (11, 20), (12, 12), (14, 16), (15, 26), (21, 19), (24, 34), (45, 55);
4.2) Online data enhancement
The data enhancement is carried out on the input image, the data set is expanded, and the main method of the data enhancement is as follows:
a. Random mirror inversion
carrying out random mirror image overturning on an input image;
b. random addition of rectangular noise
information loss in a plurality of rectangular areas with selectable area sizes and random positions in an input image is converted, information loss of all channels generates black rectangular blocks, and information loss of partial channels generates color noise;
c. Randomly adjusting contrast
Modifying hue and saturation to realize contrast conversion;
4.3) setting training completion flag
setting verification set interval detection training accuracy, setting a training completion flag to reach the maximum iteration number of 600, enabling the accuracy to meet 99% requirements, and storing the model structure and parameters after training is completed.
5) and outputting the dial area image by the dial positioning model, and inputting the dial area image into the dial information extraction model to obtain dial range information.
6) the dial positioning model output and the dial information extraction model output are used for obtaining the pointer position and the dial center position through an image processing technology, and the method comprises the following steps:
6.1) Dial correction
And obtaining a transformation matrix by using the dial plate central range position information and the central range real position information which are output by the dial plate information extraction model, and correcting the dial plate image through affine transformation.
6.2) pointer location segmentation
Obtaining a binarization dial image by applying an OTSU self-adaptive threshold segmentation method, searching a maximum contour meeting the specified range length-width ratio and area on the binarization image to obtain a pointer initial position, and obtaining the pointer position through Hough line detection and least square fitting.
6.3) Dial centering
And obtaining the center of the dial to be selected by using Hough circle detection, calculating the Euclidean distance between each center and the pointer, and screening according to the minimum distance principle to obtain the center position of the dial.
7) obtaining a total range angle from a maximum range position and a minimum range position obtained in the output of the dial information extraction model, obtaining a reading angle from a pointer position and the minimum range position, and obtaining the meter reading by an angle method which is a same proportion relation, as shown in fig. 4, the formula of the angle method is as follows:
Where num is the pointer meter reading, angle1 is the angle between the maximum range and the minimum range, angle2 is the angle between the minimum range and the pointer position, max is the maximum range reading, and min is the minimum range reading.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that variations based on the shape and principle of the present invention should be covered within the scope of the present invention.
Claims (7)
1. a pointer instrument reading identification method based on a neural network is characterized by comprising the following steps:
1) Constructing and preprocessing a dial positioning data set;
2) Designing a matched neural network model aiming at the characteristics of the identified object and the application scene;
3) Training the designed neural network model loading training parameters, performing online data enhancement in the training process, and obtaining a dial plate positioning model after the training is finished;
4) Inputting an image to be recognized into a trained dial plate positioning model to obtain the position and the category of a dial plate in the image, outputting and constructing a dial plate information extraction data set by the dial plate positioning model and preprocessing, loading training parameters on the neural network model designed in the step 2) and training on the dial plate information extraction data set, using online data enhancement in the training process, and obtaining a dial plate information extraction model after the training is finished;
5) outputting and inputting the dial plate positioning model into the trained dial plate information extraction model to obtain dial plate range information;
6) Obtaining the position of a pointer and the central position of the dial by the output of the dial positioning model and the output of the dial information extraction model through an image processing technology;
7) Obtaining a maximum range position and a minimum range position from the output of the dial information extraction model so as to obtain a total range angle, obtaining a reading angle from the pointer position and the minimum range position, and obtaining the reading of the instrument by using the same proportional relation, namely an angle method; wherein, the formula of the angle method is as follows:
where num is the pointer meter reading, angle1 is the angle between the maximum range and the minimum range, angle2 is the angle between the minimum range and the pointer position, max is the maximum range reading, and min is the minimum range reading.
2. the method for recognizing the reading of the pointer instrument based on the neural network as claimed in claim 1, wherein in step 1), image data of the pointer instrument under different scenes are collected through an image collection device, an original data set is constructed, then interference data influencing training and recognition of the neural network, including fuzzy data, extreme angle data and dial plate missing data, are removed, and the rest data are labeled, wherein the labeled content is the position and the category of the dial plate of the instrument.
3. The method for recognizing the reading of the pointer instrument based on the neural network as claimed in claim 1, wherein in the step 2), a matched neural network is constructed by combining the characteristics of the recognition object and the application scene, and the method comprises the following steps:
2.1) constructing a feature extraction network
the method comprises the following steps of constructing a feature extraction network according to real-time and high-precision requirements, wherein the feature extraction network mainly comprises a plurality of combined convolution modules and has the structure as follows:
The first layer is a combined convolution module A which consists of a zero filling layer, a convolution layer, a batch normalization layer and an activation layer;
The second layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The third layer is a combined convolution module C which consists of a zero filling layer, a deep convolution layer, two batch normalization layers, two activation layers and a convolution layer;
the fourth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
the fifth layer is a combined convolution module C which consists of a zero filling layer, a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The sixth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
the seventh layer is a combined convolution module C which consists of a zero filling layer, a deep convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The eighth layer is a combined convolution module D which consists of five combined convolution modules B;
the ninth layer is a combined convolution module C which consists of a zero filling layer, a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The tenth layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
2.2) building a prediction network
according to the output of different layers of the feature extraction network, a prediction network for outputting and predicting targets with different sizes is constructed, and the method comprises the following steps:
a. large-size target prediction network
The input is the tenth layer output of the feature extraction network, the large-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the structure of the large-size target prediction network is as follows:
The first layer is a combined convolution module D which consists of five combined convolution modules B;
The second layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The third layer is a convolution layer;
b. Medium size target prediction network
The input is the eighth layer output of the feature extraction network and the first layer output of the large-size target prediction network, the medium-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the structure of the medium-size target prediction network is as follows:
the first layer is an input fusion module which consists of a combined convolution module B, an up-sampling layer and a tensor splicing layer;
The second layer is a combined convolution module D which consists of five combined convolution modules B;
the third layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
the fourth layer is a convolution layer;
c. small size target prediction network
The input is the sixth layer output of the feature extraction network and the second layer output of the medium-size target prediction network, the small-size target prediction network mainly comprises a plurality of combined convolution modules and convolution layers, and the small-size target prediction network has the structure that:
The first layer is an input fusion module which consists of a combined convolution module B, an up-sampling layer and a tensor splicing layer;
The second layer is a combined convolution module D which consists of five combined convolution modules B;
the third layer is a combined convolution module B which consists of a depth convolution layer, two batch normalization layers, two activation layers and a convolution layer;
The fourth layer is a convolution layer;
Finally, the output of the large-size target prediction network, the medium-size target prediction network and the small-size target prediction network passes through a non-maximum value inhibition layer to obtain the positions and the types of the predicted targets;
2.3) setting the loss function
Setting a loss function as a summation mean of a central coordinate loss function, a width and height loss function, confidence coefficient loss and a category loss function, wherein the loss function formula is as follows:
Loss＝(Loss+Loss+Loss+Loss)/numf
Wherein, Loss represents total Loss, Lossxy represents central coordinate Loss, Losswh represents width and height Loss, lossconfident represents confidence Loss, Losscls represents category Loss, and numf represents floating point number of input total number; the respective loss functions are formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
Loss＝0.5*mark*(2-w*h)*(wh-wh)
Loss＝mark*Loss(mark,c)
+(1-mark)*Loss(mark,c)*mark
Loss＝mark*Loss(cls,cls)
The center coordinate loss function is formulated as follows:
Loss＝mark*(2-w*h)*Loss(xy,xy)
wherein markobject represents whether the anchor frame has a mark bit of an object, w represents the width of the anchor frame, h represents the height of the anchor frame, Losslog represents binary cross entropy loss, xytrue represents a real center coordinate value, xypredict represents a predicted center coordinate value, whtrutrutrure represents a real width and height value, whredict represents a predicted width and height value, cpredict represents a confidence value of the prediction frame, markignore represents a mark bit of the anchor frame with an IOU smaller than a threshold value, clstrie represents a real class, and clsprredict represents a prediction class.
4. the method for recognizing the reading of the pointer instrument based on the neural network as claimed in claim 1, wherein in the step 3), training the designed neural network model comprises the following steps:
3.1) setting training parameters
setting a training optimizer as Adam, an initial learning rate of 0.001, an iteration number of 500, a batch size of 8 and K-means clustering on all labels to generate initial prior frames as (38, 29), (65, 52), (94, 87), (142, 134), (195, 69), (216, 206), (337, 320), (397, 145), (638, 569);
3.2) Online data enhancement
The data enhancement is carried out on the input image, the data set is expanded, and the main method of the data enhancement is as follows:
a. Random mirror inversion
Carrying out random mirror image overturning on an input image;
b. random additive noise
Adding a continuous single noise mask to the input image;
c. randomly adjusting contrast
Modifying hue and saturation to realize contrast conversion;
3.3) setting training completion flag
And setting verification set interval detection training accuracy, marking a training completion mark to meet the requirements of the maximum iteration number and the accuracy, and storing the model structure and parameters after the training is completed.
5. the method for recognizing the reading of the pointer instrument based on the neural network as claimed in claim 1, wherein in the step 4), the output image obtained by the dial plate positioning model is integrated into a dial plate information extraction training set, and labeling work is performed, the labeling content is range information, and the neural network model designed in the step 2) is trained on the dial plate information extraction data set, comprising the following steps:
4.1) setting training parameters
Setting a training optimizer as Adam, an initial learning rate of 0.001, an iteration number of 600, a batch size of 8 and K-means clustering on all labels to generate initial prior frames of (8, 12), (10, 16), (11, 20), (12, 12), (14, 16), (15, 26), (21, 19), (24, 34), (45, 55);
4.2) Online data enhancement
The data enhancement is carried out on the input image, the data set is expanded, and the main method of the data enhancement is as follows:
a. Random mirror inversion
carrying out random mirror image overturning on an input image;
b. random addition of rectangular noise
information loss in a plurality of rectangular areas with selectable area sizes and random positions in an input image is converted, information loss of all channels generates black rectangular blocks, and information loss of partial channels generates color noise;
c. randomly adjusting contrast
modifying hue and saturation to realize contrast conversion;
4.3) setting training completion flag
and setting verification set interval detection training accuracy, marking a training completion mark to meet the requirements of the maximum iteration number and the accuracy, and storing the model structure and parameters after the training is completed.
6. the method for recognizing the reading of the pointer instrument based on the neural network as claimed in claim 1, wherein in the step 5), the dial area image is obtained by the dial positioning model output, and the dial area image is input into the dial information extraction model to obtain the dial range information.
7. the neural network-based pointer instrument reading identification method of claim 1, wherein in step 6), the pointer position and the dial center position are obtained from the dial positioning model output and the dial information extraction model output through an image processing technology, and the method comprises the following steps:
6.1) Dial correction
Obtaining a transformation matrix by using dial plate central range position information and central range real position information which are obtained by the output of the dial plate information extraction model, and correcting a dial plate image through affine transformation;
6.2) pointer location segmentation
Obtaining a binary dial image by applying an OTSU (optical transmission unit) adaptive threshold segmentation method, searching a maximum contour meeting the length-width ratio and the area of a specified range on the binary image to obtain a pointer initial position, and obtaining the pointer position through Hough line detection and least square fitting;
6.3) Dial centering
and obtaining the center of the dial to be selected by using Hough circle detection, calculating the Euclidean distance between each center and the pointer, and obtaining the center position of the dial according to the minimum distance principle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910724076.1A CN110543878B (en) | 2019-08-07 | 2019-08-07 | Pointer instrument reading identification method based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910724076.1A CN110543878B (en) | 2019-08-07 | 2019-08-07 | Pointer instrument reading identification method based on neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110543878A true CN110543878A (en) | 2019-12-06 |
| CN110543878B CN110543878B (en) | 2022-03-29 |
Family
ID=68710075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910724076.1A Active CN110543878B (en) | 2019-08-07 | 2019-08-07 | Pointer instrument reading identification method based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110543878B (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127457A (en) * | 2019-12-25 | 2020-05-08 | 上海找钢网信息科技股份有限公司 | Reinforcing steel bar number statistical model training method, statistical method, device and equipment |
| CN111368673A (en) * | 2020-02-26 | 2020-07-03 | 华南理工大学 | Method for quickly extracting human body key points based on neural network |
| CN111368825A (en) * | 2020-02-25 | 2020-07-03 | 华南理工大学 | Pointer positioning method based on semantic segmentation |
| CN111382738A (en) * | 2020-02-11 | 2020-07-07 | 济南信通达电气科技有限公司 | Reading method and device of pointer type instrument |
| CN111401357A (en) * | 2020-02-12 | 2020-07-10 | 杭州电子科技大学 | Pointer instrument reading method based on text detection |
| CN111401358A (en) * | 2020-02-25 | 2020-07-10 | 华南理工大学 | Instrument dial plate correction method based on neural network |
| CN111724338A (en) * | 2020-03-05 | 2020-09-29 | 中冶赛迪重庆信息技术有限公司 | Turntable abnormity identification method, system, electronic equipment and medium |
| CN111797909A (en) * | 2020-06-22 | 2020-10-20 | 上海工程技术大学 | Pointer instrument reading method based on machine vision |
| CN111950396A (en) * | 2020-07-27 | 2020-11-17 | 江苏大学 | Instrument reading neural network identification method |
| CN111950605A (en) * | 2020-07-28 | 2020-11-17 | 北京恒通智控机器人科技有限公司 | Meter identification model learning method, device and equipment and meter identification method |
| CN112115896A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Instrument panel pointer reading prediction method and device, computer equipment and storage medium |
| CN112464986A (en) * | 2020-10-30 | 2021-03-09 | 济南浪潮高新科技投资发展有限公司 | Reading method and device for pointer type disc instrument |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112528058A (en) * | 2020-11-23 | 2021-03-19 | 西北工业大学 | Fine-grained image classification method based on image attribute active learning |
| CN112541576A (en) * | 2020-12-14 | 2021-03-23 | 四川翼飞视科技有限公司 | Biological living body recognition neural network of RGB monocular image and construction method thereof |
| CN112712114A (en) * | 2020-12-29 | 2021-04-27 | 重庆川仪自动化股份有限公司 | Instrument analysis method, system, equipment and medium based on TextCNN-BilSTM |
| CN113029220A (en) * | 2021-01-28 | 2021-06-25 | 开放智能机器（上海）有限公司 | State recognition system and method for industrial instrument panel |
| CN113312941A (en) * | 2020-02-26 | 2021-08-27 | 北京同邦卓益科技有限公司 | Method and device for monitoring instrument panel |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113673508A (en) * | 2021-07-28 | 2021-11-19 | 华南理工大学 | Pointer instrument image data synthesis method |
| CN113673509A (en) * | 2021-07-28 | 2021-11-19 | 华南理工大学 | Instrument detection and classification method based on image text |
| CN114638961A (en) * | 2022-03-28 | 2022-06-17 | 北京国电瑞源科技发展有限公司 | Pointer dial plate identification method, system and computer storage medium |
| CN114663866A (en) * | 2022-03-30 | 2022-06-24 | 河南科技大学 | Instrument reading identification method and device based on machine vision |
| CN117173690A (en) * | 2023-10-24 | 2023-12-05 | 四川泓宝润业工程技术有限公司 | Method and device for automatically positioning and reading natural gas meter, and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809179A (en) * | 2014-12-31 | 2016-07-27 | 中国科学院深圳先进技术研究院 | Pointer type instrument reading recognition method and device |
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机系统工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109583324A (en) * | 2018-11-12 | 2019-04-05 | 武汉大学 | A kind of pointer meters reading automatic identifying method based on the more box detectors of single-point |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
-
2019
- 2019-08-07 CN CN201910724076.1A patent/CN110543878B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809179A (en) * | 2014-12-31 | 2016-07-27 | 中国科学院深圳先进技术研究院 | Pointer type instrument reading recognition method and device |
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机系统工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| CN109583324A (en) * | 2018-11-12 | 2019-04-05 | 武汉大学 | A kind of pointer meters reading automatic identifying method based on the more box detectors of single-point |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
Non-Patent Citations (2)
| Title |
|---|
| LIUCHEN QIAN ET AL.: ""An Efficient Model Compression Method for CNN Based Object Detection"", 《 2018 IEEE 9TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS)》 * |
| 王瑞 等: ""一种基于改进角度法的指针式仪表图像自动读数方法"", 《电测与仪表》 * |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127457A (en) * | 2019-12-25 | 2020-05-08 | 上海找钢网信息科技股份有限公司 | Reinforcing steel bar number statistical model training method, statistical method, device and equipment |
| CN111382738A (en) * | 2020-02-11 | 2020-07-07 | 济南信通达电气科技有限公司 | Reading method and device of pointer type instrument |
| CN111401357B (en) * | 2020-02-12 | 2023-09-15 | 杭州电子科技大学 | Pointer type instrument reading method based on text detection |
| CN111401357A (en) * | 2020-02-12 | 2020-07-10 | 杭州电子科技大学 | Pointer instrument reading method based on text detection |
| CN111368825A (en) * | 2020-02-25 | 2020-07-03 | 华南理工大学 | Pointer positioning method based on semantic segmentation |
| CN111401358A (en) * | 2020-02-25 | 2020-07-10 | 华南理工大学 | Instrument dial plate correction method based on neural network |
| CN111368825B (en) * | 2020-02-25 | 2023-04-18 | 华南理工大学 | Pointer positioning method based on semantic segmentation |
| CN111401358B (en) * | 2020-02-25 | 2023-05-09 | 华南理工大学 | Instrument dial correction method based on neural network |
| CN113312941A (en) * | 2020-02-26 | 2021-08-27 | 北京同邦卓益科技有限公司 | Method and device for monitoring instrument panel |
| CN111368673A (en) * | 2020-02-26 | 2020-07-03 | 华南理工大学 | Method for quickly extracting human body key points based on neural network |
| CN111368673B (en) * | 2020-02-26 | 2023-04-07 | 华南理工大学 | Method for quickly extracting human body key points based on neural network |
| CN111724338B (en) * | 2020-03-05 | 2023-04-18 | 中冶赛迪信息技术(重庆)有限公司 | Turntable abnormity identification method, system, electronic equipment and medium |
| CN111724338A (en) * | 2020-03-05 | 2020-09-29 | 中冶赛迪重庆信息技术有限公司 | Turntable abnormity identification method, system, electronic equipment and medium |
| CN111797909B (en) * | 2020-06-22 | 2024-03-29 | 上海工程技术大学 | Pointer type instrument reading method based on machine vision |
| CN111797909A (en) * | 2020-06-22 | 2020-10-20 | 上海工程技术大学 | Pointer instrument reading method based on machine vision |
| CN111950396B (en) * | 2020-07-27 | 2024-05-14 | 江苏大学 | Meter reading neural network identification method |
| CN111950396A (en) * | 2020-07-27 | 2020-11-17 | 江苏大学 | Instrument reading neural network identification method |
| CN111950605A (en) * | 2020-07-28 | 2020-11-17 | 北京恒通智控机器人科技有限公司 | Meter identification model learning method, device and equipment and meter identification method |
| CN112115896A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Instrument panel pointer reading prediction method and device, computer equipment and storage medium |
| CN112115896B (en) * | 2020-09-24 | 2023-12-22 | 深圳市赛为智能股份有限公司 | Instrument panel pointer reading prediction method and device, computer equipment and storage medium |
| CN112464986A (en) * | 2020-10-30 | 2021-03-09 | 济南浪潮高新科技投资发展有限公司 | Reading method and device for pointer type disc instrument |
| CN112528058A (en) * | 2020-11-23 | 2021-03-19 | 西北工业大学 | Fine-grained image classification method based on image attribute active learning |
| CN112528058B (en) * | 2020-11-23 | 2022-09-02 | 西北工业大学 | Fine-grained image classification method based on image attribute active learning |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112541576A (en) * | 2020-12-14 | 2021-03-23 | 四川翼飞视科技有限公司 | Biological living body recognition neural network of RGB monocular image and construction method thereof |
| CN112541576B (en) * | 2020-12-14 | 2024-02-20 | 四川翼飞视科技有限公司 | Biological living body identification neural network construction method of RGB monocular image |
| CN112712114A (en) * | 2020-12-29 | 2021-04-27 | 重庆川仪自动化股份有限公司 | Instrument analysis method, system, equipment and medium based on TextCNN-BilSTM |
| CN112712114B (en) * | 2020-12-29 | 2024-05-14 | 重庆川仪自动化股份有限公司 | Method, system, equipment and medium for analyzing instrument based on TextCNN-BiLSTM |
| CN113029220A (en) * | 2021-01-28 | 2021-06-25 | 开放智能机器（上海）有限公司 | State recognition system and method for industrial instrument panel |
| CN113469178B (en) * | 2021-07-05 | 2024-03-01 | 安徽南瑞继远电网技术有限公司 | Power meter identification method based on deep learning |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113673508B (en) * | 2021-07-28 | 2023-05-26 | 华南理工大学 | Pointer instrument image data synthesis method |
| CN113673509B (en) * | 2021-07-28 | 2023-06-09 | 华南理工大学 | Instrument detection classification method based on image text |
| CN113673509A (en) * | 2021-07-28 | 2021-11-19 | 华南理工大学 | Instrument detection and classification method based on image text |
| CN113673508A (en) * | 2021-07-28 | 2021-11-19 | 华南理工大学 | Pointer instrument image data synthesis method |
| CN114638961A (en) * | 2022-03-28 | 2022-06-17 | 北京国电瑞源科技发展有限公司 | Pointer dial plate identification method, system and computer storage medium |
| CN114663866A (en) * | 2022-03-30 | 2022-06-24 | 河南科技大学 | Instrument reading identification method and device based on machine vision |
| CN117173690A (en) * | 2023-10-24 | 2023-12-05 | 四川泓宝润业工程技术有限公司 | Method and device for automatically positioning and reading natural gas meter, and electronic equipment |
| CN117173690B (en) * | 2023-10-24 | 2024-01-26 | 四川泓宝润业工程技术有限公司 | Method and device for automatically positioning and reading natural gas meter, and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110543878B (en) | 2022-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110543878B (en) | Pointer instrument reading identification method based on neural network | |
| CN110070008B (en) | Bridge disease identification method adopting unmanned aerial vehicle image | |
| CN113673509B (en) | Instrument detection classification method based on image text | |
| CN112329588B (en) | Pipeline fault detection method based on Faster R-CNN | |
| CN111401358B (en) | Instrument dial correction method based on neural network | |
| CN111368825B (en) | Pointer positioning method based on semantic segmentation | |
| CN110222767B (en) | Three-dimensional point cloud classification method based on nested neural network and grid map | |
| CN114549981A (en) | Intelligent inspection pointer type instrument recognition and reading method based on deep learning | |
| CN112613097A (en) | BIM rapid modeling method based on computer vision | |
| CN109215015A (en) | A kind of online visible detection method of silk cocoon based on convolutional neural networks | |
| CN115862073B (en) | Substation hazard bird species target detection and identification method based on machine vision | |
| CN110263790A (en) | A kind of power plant's ammeter character locating and recognition methods based on convolutional neural networks | |
| CN115272204A (en) | Bearing surface scratch detection method based on machine vision | |
| CN116612106A (en) | Method for detecting surface defects of optical element based on YOLOX algorithm | |
| CN111539330B (en) | Transformer substation digital display instrument identification method based on double-SVM multi-classifier | |
| CN111369526A (en) | Multi-type old bridge crack identification method based on semi-supervised deep learning | |
| CN114549446A (en) | Cylinder sleeve defect mark detection method based on deep learning | |
| CN115861170A (en) | Surface defect detection method based on improved YOLO V4 algorithm | |
| CN110321890B (en) | Digital instrument identification method of power inspection robot | |
| CN113705564A (en) | Pointer type instrument identification reading method | |
| CN111368637B (en) | Transfer robot target identification method based on multi-mask convolutional neural network | |
| CN110569764B (en) | Mobile phone model identification method based on convolutional neural network | |
| CN117197146A (en) | Automatic identification method for internal defects of castings | |
| CN112529003A (en) | Instrument panel digital identification method based on fast-RCNN | |
| CN117152727A (en) | Automatic reading method of pointer instrument for inspection robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |