CN113780310A - Instrument reading method based on key point detection - Google Patents
Instrument reading method based on key point detection Download PDFInfo
- Publication number
- CN113780310A CN113780310A CN202111039693.1A CN202111039693A CN113780310A CN 113780310 A CN113780310 A CN 113780310A CN 202111039693 A CN202111039693 A CN 202111039693A CN 113780310 A CN113780310 A CN 113780310A
- Authority
- CN
- China
- Prior art keywords
- pointer
- instrument
- reading
- point
- angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 title claims abstract description 37
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000013507 mapping Methods 0.000 claims abstract description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 4
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 11
- 230000009466 transformation Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
Abstract
The invention relates to the field of image processing, in particular to a meter reading method based on key point detection, which comprises the following steps: establishing an instrument key point detection training model; acquiring an instrument image; acquiring a pointer tip point and a center point of the pointer instrument based on the training model and the instrument image; the angle of the pointer can be calculated by using the point of the pointer tip and the center point; and acquiring the reading of the pointer instrument according to the angle-reading mapping table. The invention abandons the mode of directly using binaryzation to detect the pointer of the instrument, and uses the mode of key point detection based on deep learning to detect two key points of the pointer instrument, namely the tip of the pointer and the central point of the pointer instrument. The angle of the pointer can be calculated by using the two key points, and then the reading of the pointer instrument at the moment is obtained according to the angle-reading mapping of each range, so that the robustness of the algorithm and the accuracy of the reading are improved.
Description
Technical Field
The invention relates to the field of image processing, in particular to a meter reading method based on key point detection.
Background
In areas such as automated factories, intensive instrument laboratories, and hospital radiation areas, it is often necessary to evaluate the operation conditions of different devices in these environments by the readings of the meters, and if manual readings are taken, the health of workers will be greatly injured, so that it is necessary to use an intelligent manner instead of manually reading the readings of the meters.
In the field of meter reading, the directions of research are roughly divided into two main categories. The other is to research an intelligent instrument, and the intelligent device for automatically reading reports from the hardware perspective only needs to operate hardware to display data when the reading of the instrument needs to be known. However, hardware optimization is relatively costly and requires additional maintenance over a long period of time. And secondly, the reading function of the pointer instrument is realized by researching a proper and efficient method, so that the complicated manual operation steps are simplified. In some existing meter reading algorithms, an image binarization method is mostly adopted, and although the processing method for directly binarizing the image is simple, the method has the disadvantages of being very easy to be influenced by background factors and very sensitive to illumination factors. Besides direct binarization processing, a method similar to template matching and pointer rotation is adopted to detect the reading of a pointer instrument, the method is also greatly influenced by background factors of the instrument, and once background patterns are more, algorithm misjudgment is easy to cause the reading result to be greatly different from a real result. In addition to background factors, the increment of rotation of the pointer mask is also a very critical factor, if the increment of rotation is too small, the time taken for each reading is too long, and if the increment of rotation is too large, the time taken for reading is reduced, but the increment of rotation is too large, so that the reading result is very large in error.
Disclosure of Invention
The invention aims to provide a meter reading method based on key point detection, and aims to solve the problem that the reliability of reading is greatly reduced due to the fact that the existing pointer meter reading method is easily influenced by environment and illumination.
In order to achieve the above object, the present invention provides a method for reading a meter based on key point detection, comprising: establishing an instrument key point detection training model; acquiring an instrument image; acquiring a pointer tip point and a center point of the pointer instrument based on the training model and the instrument image; the angle of the pointer can be calculated by using the point of the pointer tip and the center point; and acquiring the reading of the pointer instrument according to the angle-reading mapping table.
After the instrument image is obtained and before the pointer tip point and the center point of the pointer instrument are obtained based on the training model and the instrument image, the method further comprises the steps of judging whether the instrument image is inclined or not, and correcting deviation if the instrument image is inclined.
The specific steps for establishing the instrument key point detection training model are as follows:
establishing a neural network training data set;
marking the pointer tip point and the center point through manual marking software to generate a label file corresponding to the pointer instrument;
transmitting the label file into a Mask R-CNN network for iterative training to obtain a final training model;
and evaluating the detection effect of the training model by using a standard mean square error function.
The specific steps of transmitting the label file into a Mask R-CNN network for iterative training to obtain a final training model are as follows:
changing the mask branch structure into a key point branch structure;
confirming a loss function;
and performing iterative training based on the label file.
The key point branch structure is composed of two groups of convolution layers, wherein the size of the first group of convolution layers is 14x14x256, and the size of the second group of convolution layers is 14x14x 2.
The specific steps of calculating the angle of the pointer by using the pointer tip point and the center point comprise:
vertically making a line at the center of the pointer instrument, taking the line as an area dividing line, and setting the pointer instrument to point to the left of the area dividing line when the reading number is 0;
converting the coordinates with the upper left corner as an origin in the instrument image into coordinates with the central point of the image as the origin of the coordinates;
acquiring a quadrant where the pointer is located;
and calculating the inclination angle of the pointer by combining the quadrant in which the pointer is positioned.
The specific steps of obtaining the reading of the pointer instrument according to the angle-reading mapping table are as follows:
constructing a plurality of interval angle-reading mapping dictionaries;
judging the interval pointed by the pointer according to the inclination angle, and calculating the precision according to the upper limit and the lower limit of the interval;
and outputting the reading of the pointer instrument.
The invention discloses a meter reading method based on key point detection, which comprises the following steps: establishing an instrument key point detection training model; acquiring an instrument image; acquiring a pointer tip point and a center point of the pointer instrument based on the training model and the instrument image; the angle of the pointer can be calculated by using the point of the pointer tip and the center point; and acquiring the reading of the pointer instrument according to the angle-reading mapping table. The invention abandons the mode of directly using binaryzation to detect the pointer of the instrument, and uses the mode of key point detection based on deep learning to detect two key points of the pointer instrument, namely the tip of the pointer and the central point of the pointer instrument. The angle of the pointer can be calculated by using the two key points, and then the reading of the pointer instrument at the moment is obtained according to the angle-reading mapping of each range, so that the robustness of the algorithm and the accuracy of the reading are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is an exemplary diagram of a single pointer meter;
FIG. 2 is a diagram of a network header structure after modification of a key point branch;
FIG. 3 is a logic diagram of a method of meter reading based on keypoint detection in accordance with the present invention;
FIG. 4 is a flow chart of a method of meter reading based on keypoint detection in accordance with the present invention;
FIG. 5 is a flow chart of the present invention for building an instrument keypoint detection training model;
FIG. 6 is a flowchart of the present invention for transferring a label file into a Mask R-CNN network for iterative training to obtain a final training model;
FIG. 7 is a flow chart of the present invention for calculating the angle of the pointer using the point of the pointer tip and the center point;
FIG. 8 is a flow chart of the present invention for obtaining a reading of a pointer meter from an angle-reading mapping table.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
Referring to fig. 1 to 8, the present invention provides a method for reading a meter based on a key point detection, including:
s101, establishing an instrument key point detection training model;
take a single pointer meter as an example.
The method comprises the following specific steps:
s201, establishing a neural network training data set;
a neural network training data set is established for the single pointer instrument.
S202, marking a pointer tip point and a center point through manual marking software to generate a label file corresponding to the pointer instrument;
the key points of the single-pointer instrument are labeled through manual labeling software, and for the single-pointer instrument, the key points are the center point P of the pointer instrument, and the point Q of the tip of the pointer is shown in FIG. 1. And generating a label file corresponding to the pointer instrument after labeling.
S203, transmitting the label file into a Mask R-CNN network for iterative training to obtain a final training model;
the method comprises the following specific steps:
s301, changing a mask branch structure into a key point branch structure;
in the process of training the model of the instrument reading method based on the key point detection, the head part of the model needs to be modified into a head part suitable for the key point detection, that is, the original mask branch structure is changed into the key point branch structure shown in fig. 2, the structure is composed of two groups of convolutional layers, the size of the first group of convolutional layers is 14x14x256, the size of the second group of convolutional layers is 14x14x2, and the third dimension of the second group of convolutional layers is 2 because only two key points of the pointer instrument are considered, that is, the point P and the point Q in fig. 1.
S302, confirming a loss function;
after the head structure is modified, a loss function needs to be confirmed, and the loss function used for training the example segmentation task in the original paper is as follows:
L＝Lcls+Lbox+Lmask
in the model training process of the instrument reading method based on the key point detection, a loss function needs to be modified into the following function:
L＝Lcls+Lbox+Lkpt
wherein L isclsAnd LboxSplitting L in task with original exampleclsAnd LboxIdentity, LclsRepresents a category loss term, LboxRepresenting the prediction box coordinate parameter penalty term. Modifying item LkptThe expression of (a) is:
wherein k is the number of key points, and since the number of key points of the pointer instrument is 2, the value of k is 1 and 2. Let H be the height of the image, W be the width of the image, a ═ 1, 2, 3,. · H }, B ═ 1, 2, 3,.. · W }, p denote the probability distribution of each region of interest divided by k +1 classes.
S303 performs iterative training based on the label file.
S204, the detection effect of the training model is evaluated by using a standard mean square error function.
The expression of the standard mean square error estimator function (NE) used is as follows:
where k represents the kth keypoint. dkIs the euclidean distance between the location of the predicted keypoint and the location of the true keypoint. skIs a distance normalization parameter, which is the radius length of the pointer instrument for the pointer instrument. v. ofkIs the visibility of the key points, and the visibility of both key points is set to 1 in the present invention. If the evaluation result reaches the expected effect, the training effect of the model is satisfied, and the trained model is used for detecting key points of the instrument,
s102, acquiring an instrument image;
the instrument image can be acquired by taking a picture.
S103, judging whether the instrument image is inclined or not, and if so, correcting the deviation;
before the trained model is used for detecting key points of the instrument, whether the instrument is inclined or not due to the shooting angle needs to be judged, and if the instrument is inclined, perspective transformation is carried out to correct the pointer instrument image.
The general process of correcting pointer meters based on perspective transformation of the meter reading method of keypoint detection is as follows:
is provided with [ X Y Z]TIs the original view plane, [ x y 1 ]]TIs the target view plane, the perspective transformation matrix is:
the backbone perspective transformation equation can perform perspective transformation on the original view plane:
s104, acquiring a pointer tip point and a center point of the pointer instrument based on the training model and the instrument image;
the instrument image obtained in step S103 after being subjected to perspective transformation and correction is transmitted to a model for prediction, that is, the positions of the pointer tip point and the center point can be obtained, if there are a plurality of pointer instruments in the image, the model can predict the center point and the pointer tip point in a plurality of groups of instruments, the center point in each group is matched with each pointer tip point to generate a plurality of groups of detection objects, and thus, the problem of the multi-pointer instrument can be converted into the problem of a plurality of single-pointer instruments for processing.
S105, combining the pointer tip point and the center point to calculate the angle of the pointer;
the method comprises the following specific steps:
s401, a line is vertically made in the center of the pointer instrument, the line is used as an area dividing line, and when the reading number of the pointer instrument is set to be 0, the pointer points to the left side of the area dividing line;
s402, converting the coordinates with the upper left corner as an origin in the instrument image into coordinates with the central point of the image as the origin of the coordinates;
where (x ', y') is the coordinates before conversion, and (x, y) is the coordinates after conversion.
S403, acquiring a quadrant where the pointer is located;
it is necessary to judge that the pointer points to the quadrant number before the calculation of the angle is performed. According to the analysis of various conditions, the following judgment criteria are obtained:
wherein (x)P，yP) And (x)Q，yQ) The key points P and Q are coordinates in the original image with the center point of the image as the origin of coordinates.
S404, calculating the inclination angle of the pointer by combining the quadrant where the pointer is located;
in conjunction with the quadrant of the pointer, the angle θ of the pointer at this time with respect to the horizontal right x-axis is calculated:
s106, reading of the pointer instrument is obtained according to the angle-reading mapping table.
The method comprises the following specific steps:
s501, constructing a plurality of interval angle-reading mapping dictionaries;
an angle-reading mapping dictionary of the instrument reading method based on the key point detection is constructed according to the pointer instrument in fig. 1, as shown in table 1:
TABLE 1
S502, judging the interval pointed by the pointer according to the inclination angle, and calculating the precision according to the upper limit and the lower limit of the interval;
and judging the interval pointed by the pointer according to the inclination angle, and calculating the precision phi according to the upper limit and the lower limit of the interval as follows:
wherein b and a are the upper and lower limits of the pointer angle of the interval, rbAnd raRespectively, the reading when the pointer points to the upper and lower limits.
S503 outputs the reading of the pointer meter.
The reading R of the pointer instrument at the moment is output as follows:
R＝φ*(θ-a)
the invention abandons the mode of directly using binaryzation to detect the pointer of the instrument, and uses the mode of key point detection based on deep learning to detect two key points of the pointer instrument, namely the tip of the pointer and the central point of the pointer instrument. The angle of the pointer can be calculated by using the two key points, and then the reading of the pointer instrument at the moment is obtained according to the angle-reading mapping of each range, so that the robustness of the algorithm and the accuracy of the reading are improved.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (7)
1. A meter reading method based on key point detection is characterized in that,
the method comprises the following steps: establishing an instrument key point detection training model;
acquiring an instrument image;
acquiring a pointer tip point and a center point of the pointer instrument based on the training model and the instrument image;
the angle of the pointer can be calculated by using the point of the pointer tip and the center point;
and acquiring the reading of the pointer instrument according to the angle-reading mapping table.
2. The method of claim 1, wherein the key point detection-based meter reading method,
after the instrument image is obtained and before the pointer tip point and the center point of the pointer instrument are obtained based on the training model and the instrument image, the method further comprises the steps of judging whether the instrument image is inclined or not, and correcting deviation if the instrument image is inclined.
3. The method of claim 1, wherein the key point detection-based meter reading method,
the specific steps of establishing the instrument key point detection training model are as follows:
establishing a neural network training data set;
marking the pointer tip point and the center point through manual marking software to generate a label file corresponding to the pointer instrument;
transmitting the label file into a MaskR-CNN network for iterative training to obtain a final training model;
and evaluating the detection effect of the training model by using a standard mean square error function.
4. A method of reading a meter based on keypoint detection according to claim 3,
the specific steps of transmitting the label file into a MaskR-CNN network for iterative training to obtain a final training model are as follows:
changing the mask branch structure into a key point branch structure;
confirming a loss function;
and performing iterative training based on the label file.
5. The method of claim 4 wherein the key point detection based meter reading method,
the critical point branching structure is composed of two groups of convolution layers, wherein the size of the first group of convolution layers is 14x14x256, and the size of the second group of convolution layers is 14x14x 2.
6. The method of claim 1, wherein the key point detection-based meter reading method,
the specific steps of calculating the angle of the pointer by using the point and the center point of the pointer tip comprise:
vertically making a line at the center of the pointer instrument, taking the line as an area dividing line, and setting the pointer instrument to point to the left of the area dividing line when the reading number is 0;
converting the coordinates with the upper left corner as an origin in the instrument image into coordinates with the central point of the image as the origin of the coordinates;
acquiring a quadrant where the pointer is located;
and calculating the inclination angle of the pointer by combining the quadrant in which the pointer is positioned.
7. The method of claim 6, wherein the key point detection-based meter reading method,
the specific steps of obtaining the reading of the pointer instrument according to the angle-reading mapping table are as follows:
constructing a plurality of interval angle-reading mapping dictionaries;
judging the interval pointed by the pointer according to the inclination angle, and calculating the precision according to the upper limit and the lower limit of the interval;
and outputting the reading of the pointer instrument.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111039693.1A CN113780310A (en) | 2021-09-06 | 2021-09-06 | Instrument reading method based on key point detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111039693.1A CN113780310A (en) | 2021-09-06 | 2021-09-06 | Instrument reading method based on key point detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113780310A true CN113780310A (en) | 2021-12-10 |
Family
ID=78841173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111039693.1A Pending CN113780310A (en) | 2021-09-06 | 2021-09-06 | Instrument reading method based on key point detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113780310A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909738A (en) * | 2019-11-15 | 2020-03-24 | 杭州远鉴信息科技有限公司 | Automatic reading method of pointer instrument based on key point detection |
| CN111553345A (en) * | 2020-04-22 | 2020-08-18 | 上海浩方信息技术有限公司 | Method for realizing meter pointer reading identification processing based on Mask RCNN and orthogonal linear regression |
| CN111582071A (en) * | 2020-04-23 | 2020-08-25 | 浙江大学 | SF6 instrument image reading method based on HRNet network model |
| CN112257676A (en) * | 2020-11-19 | 2021-01-22 | 南京天创电子技术有限公司 | Pointer instrument reading method and system and inspection robot |
| CN112801094A (en) * | 2021-02-02 | 2021-05-14 | 中国长江三峡集团有限公司 | Pointer instrument image inclination correction method |
| CN112949564A (en) * | 2021-02-02 | 2021-06-11 | 电子科技大学 | Pointer type instrument automatic reading method based on deep learning |
-
2021
- 2021-09-06 CN CN202111039693.1A patent/CN113780310A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909738A (en) * | 2019-11-15 | 2020-03-24 | 杭州远鉴信息科技有限公司 | Automatic reading method of pointer instrument based on key point detection |
| CN111553345A (en) * | 2020-04-22 | 2020-08-18 | 上海浩方信息技术有限公司 | Method for realizing meter pointer reading identification processing based on Mask RCNN and orthogonal linear regression |
| CN111582071A (en) * | 2020-04-23 | 2020-08-25 | 浙江大学 | SF6 instrument image reading method based on HRNet network model |
| CN112257676A (en) * | 2020-11-19 | 2021-01-22 | 南京天创电子技术有限公司 | Pointer instrument reading method and system and inspection robot |
| CN112801094A (en) * | 2021-02-02 | 2021-05-14 | 中国长江三峡集团有限公司 | Pointer instrument image inclination correction method |
| CN112949564A (en) * | 2021-02-02 | 2021-06-11 | 电子科技大学 | Pointer type instrument automatic reading method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112906694B (en) | Reading correction system and method for transformer substation inclined pointer instrument image | |
| CN109948469B (en) | Automatic inspection robot instrument detection and identification method based on deep learning | |
| CN112766103B (en) | Machine room inspection method and device | |
| CN110826549A (en) | Inspection robot instrument image identification method and system based on computer vision | |
| CN112801094B (en) | Pointer instrument image inclination correction method | |
| CN111488874A (en) | Method and system for correcting inclination of pointer instrument | |
| CN114549835A (en) | Pointer instrument correction identification method and device based on deep learning | |
| CN103295239A (en) | Laser-point cloud data automatic registration method based on plane base images | |
| CN111598942A (en) | Method and system for automatically positioning electric power facility instrument | |
| CN111368906A (en) | Pointer type oil level indicator reading identification method based on deep learning | |
| CN110211178B (en) | Pointer instrument identification method using projection calculation | |
| CN110909738A (en) | Automatic reading method of pointer instrument based on key point detection | |
| CN109508709B (en) | Single pointer instrument reading method based on machine vision | |
| CN114266881A (en) | Pointer type instrument automatic reading method based on improved semantic segmentation network | |
| CN113947714B (en) | Multi-mode collaborative optimization method and system for video monitoring and remote sensing | |
| CN113780310A (en) | Instrument reading method based on key point detection | |
| CN115661446A (en) | Pointer instrument indication automatic reading system and method based on deep learning | |
| CN115512343A (en) | Method for correcting and recognizing reading of circular pointer instrument | |
| CN112215240B (en) | Optimization method for improving 2D complex edge detection precision | |
| CN111462195B (en) | Irregular angle direction cost aggregation path determination method based on dominant line constraint | |
| CN114255458A (en) | Method and system for identifying reading of pointer instrument in inspection scene | |
| CN109470290B (en) | Automatic calibration method and device for instrument pointer | |
| CN113989513A (en) | Method for recognizing reading of square pointer type instrument | |
| CN113076941A (en) | Single pointer dial reading identification method based on video frame fusion | |
| CN112417975A (en) | Method and device for obtaining reading of pointer instrument |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |