CN114241194A

CN114241194A - Instrument identification and reading method based on lightweight network

Info

Publication number: CN114241194A
Application number: CN202111563124.7A
Authority: CN
Inventors: 沈家全; 何奇; 李德光; 张永新; 赵中乾; 孙杰; 方天乐; 刘晓瑜
Original assignee: Luoyang Normal University
Current assignee: Luoyang Normal University
Priority date: 2021-12-20
Filing date: 2021-12-20
Publication date: 2022-03-25

Abstract

The invention discloses a light-weight network-based instrument identification and reading method, which relates to the technical field of instrument measurement and comprises the following steps: acquiring an original image with a meter; performing instrument target identification and classification on the original image to obtain a classified instrument image; identifying the classification instrument image by adopting a fine-grained classification network combined with mixed attention clipping to obtain the range information of the instrument image; obtaining a meter reading based on the range information of the meter image; the learning ability of the model to the image detail target is improved by a multi-scale feature fusion means, the arithmetic efficiency of the algorithm is improved by the calculation of the single-scale YOLO, and the detection precision of the instrument target is not reduced; moreover, automatic identification, reading and data storage of the instrument can be realized, complex manual identification and recording work is eliminated for workers, the efficiency of the workers is improved, and the utilization rate of data is improved.

Description

Instrument identification and reading method based on lightweight network

Technical Field

The invention relates to the technical field of instrument measurement, in particular to an instrument identification and reading method based on a lightweight network.

Background

The instrument is a general term for an instrument used for measuring certain physical quantities, and can visually display certain physical quantities which are difficult to visually sense through scales or numbers through some physical or chemical principles, such as temperature, pressure, humidity, flow, voltage and the like, and is widely applied to various aspects of industry, national defense, people's life and the like. Especially in some important fields of national defense and military industry, reading of instrument data usually requires a large amount of labor, and at present, most of the instrument data are stored by paper media and are difficult to further utilize.

At present, two modes are mostly adopted in the industry for instrument identification, an instrument with an electronic data interface is advanced and convenient, instrument data can be read directly through the electronic interface, but the popularization degree of the instrument is low at present, more old models which are laid in the early years still exist in an actual scene, and the instrument data needs to be read through a visual mode. Therefore, in practical situations, more ways of manually reading the meter are still adopted. However, the manual reading method has several disadvantages: under some extreme environments, such as high temperature, high pressure and high radiation conditions, the mode of manual reading is more dangerous; if a large amount of real-time instrument data needs to be acquired, a large amount of manpower is required, the efficiency is low, and the cost performance is low; long-term uninterrupted monitoring for 24 hours is difficult to carry out manually; the reading that artifical acquireed often stores through the paper media, needs further to transfer into the computer, and the utilization that can carry out next step, further improvement manpower consumes.

With the development of society, the manufacturing industry is towards intellectuality development, utilizes techniques such as artificial intelligence to promote productivity, liberates the manual work. Therefore, the two methods for identifying the meter cannot satisfy the current production method, and therefore, it is an urgent need for those skilled in the art to develop an intelligent meter identification and reading method.

Disclosure of Invention

In view of this, the invention provides a method for identifying and reading a meter based on a lightweight network, which overcomes the defects of the prior art.

In order to achieve the above purpose, the invention provides the following technical scheme:

a meter identification method based on a lightweight network comprises the following specific steps:

acquiring an original image with a meter;

performing feature extraction on an original image through a RegNet lightweight network to obtain a first extracted feature map;

fusing the features in the first extracted feature map to obtain a fused image;

and identifying and classifying instrument targets of the fused images to obtain classified instrument images.

Optionally, the acquiring step of the fused image is: and performing double up-sampling on the N stage characteristics in the RegNet lightweight network, and performing characteristic fusion on the N stage characteristics and the N-1 stage characteristics, wherein the RegNet lightweight network comprises N stages.

Optionally, the instrument target classes include square pointer instruments, round pointer instruments, and digital instruments.

Optionally, the loss function in the process of identifying and classifying the instrument target comprises positioning loss l_boxClass loss l_clsAnd confidence loss l_noobj。

A meter reading method based on a lightweight network comprises the following specific steps:

acquiring an image of the classification instrument;

identifying the classification instrument image by adopting a fine-grained classification network combined with mixed attention clipping to obtain the range information of the instrument image;

based on the range information of the meter image, a meter reading is obtained.

Optionally, the specific steps of obtaining the range information of the instrument image include:

s21, extracting the features of the classified instrument image through a feature extraction network of the fusion channel attention module to obtain a second extracted feature map;

s22, processing the features in the second extracted feature map to obtain a plurality of attention maps with position information;

s23, fusing the attention diagrams, the first extracted feature diagram and the second extracted feature diagram by utilizing bilinear attention pooling, and calculating cross entropy loss to obtain feature matrixes corresponding to the attention diagrams;

s24, based on the position information in the attention diagrams, performing mixed clipping and data enhancement on the attention diagrams, and inputting the result into the feature extraction network of the fusion channel attention module in the step S11;

and S25, determining the measuring range information of the instrument image based on the characteristic matrix.

Optionally, when the target of the meter is a square pointer meter, the method for reading the meter specifically includes:

a1, carrying out Gaussian blur and image graying processing on the classification instrument image to obtain a grayscale image;

a2, carrying out edge detection on the gray level image to obtain an edge binary image;

a3, detecting the pointer in the edge binary image by a straight line detection method, and obtaining a pointer angle by pointer angle correction;

a4, calculating the reading of the instrument according to the range information and the pointer angle of the instrument image to obtain the reading value of the instrument;

wherein, the calculation formula of the meter reading value result is as follows:

in the formula, theta₂Is the inclination angle theta of the pointer in the square pointer type instrument₁The horizontal inclination angle of the image in the square pointer instrument is shown, and the range is the measuring range information of the instrument.

Optionally, when the target of the meter is a circular pointer meter, the method for reading the meter specifically includes:

b1, carrying out Gaussian blur and image graying processing on the classification instrument image to obtain a grayscale image;

b2, processing the gray level image by a local adaptive threshold value binarization processing method to obtain a binarization image;

b3, carrying out outline detection on the binary image by adopting a Hough transform method;

b4, straightening the circular scales through coordinate transformation;

b5, determining the scale position after straightening treatment by adopting a horizontal projection mode, and determining the meter reading according to the length of the scale position;

in the formula (d)₁Length from zero scale mark to the position of the pointer, d₂Range is the total length of the scale mark and the range is the range information of the instrument.

Optionally, when the target of the meter is a digital meter, the method for reading the meter specifically includes:

c1, performing Gaussian blur on the classification instrument image, and extracting a digital part in the image by adopting a red channel-based binarization method;

c2, extracting a digital display area in the digital part by adopting a horizontal projection method;

c3, dividing the numbers in the digital display area by adopting a vertical projection method;

and C4, utilizing a Lenent-5 network to identify the segmented numbers.

Optionally, in the step C3, when there is a character to be segmented in the digital display area, a refinement algorithm is used to identify and segment the character.

According to the technical scheme, the invention discloses and provides a light-weight network-based instrument identification and reading method, and compared with the prior art, the method has the following beneficial effects:

(1) the invention provides a single-scale YOLO instrument detection method combined with a lightweight network in the instrument target and classification process, the learning capability of a model on an image detail target is improved by a multi-scale feature fusion means, the operation efficiency of an algorithm is improved by the calculation of the single-scale YOLO, and the detection precision of the instrument target is not reduced;

(2) the invention provides a fine-grained image recognition method for instrument range recognition, aiming at the overfitting possibly brought by bilinear convergence and attention discarding operation in a WS-DAN algorithm, the algorithm is combined with attention mixed cutting to carry out data enhancement, and attention peak areas and real marks in different images are exchanged, so that a target can be under different backgrounds, a model can pay more attention to a local area with discriminability, and further the generalization performance and the fine-grained image recognition accuracy of the model are enhanced. The algorithm is simple and visual, and can be expanded and applied to other algorithms.

(3) The invention integrates various image processing methods based on the instrument identification detection and range identification method, realizes the automatic identification and reading of the instrument, can support the full-automatic identification of a square pointer instrument, a circular pointer instrument and a digital instrument, and does not need manual participation in the whole process. Meanwhile, the calculation amount of the algorithm is low, the processing speed of the square pointer instrument and the digital instrument can reach about 200ms on a CPU, and meanwhile, the precision reaches a high level.

(4) The instrument identification and reading method provided by the invention realizes automatic identification, reading and data storage of the instrument, reduces complicated manual identification and recording work for workers, improves the efficiency of the workers and simultaneously improves the utilization rate of data.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a flow chart of a lightweight network-based instrument identification method of the present invention;

FIG. 2 is a schematic diagram of a method for identifying and classifying objects in a meter according to the present invention;

FIG. 3 is a schematic diagram of a light-weight RegNet network according to the present invention;

FIG. 4 is a flow chart of a reading method of the square pointer instrument of the present invention;

FIG. 5 is a flow chart of a reading method of the circular pointer instrument of the present invention;

fig. 6 is a flow chart of a reading method of the digital meter according to the invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention discloses a light-weight network-based meter reading method, which comprises the following steps of:

s1, acquiring an original image with the instrument;

s2, identifying and classifying the instrument target in the original image to obtain a classified instrument image;

s3, identifying the instrument image by adopting a fine-grained classification network combined with mixed attention clipping, and acquiring the range information of the instrument image;

and S4, reading the meter based on the measuring range information of the meter image.

As shown in fig. 2, step S2 of this embodiment specifically includes:

s21, extracting the features of the original image through a RegNet lightweight network;

s22, performing double up-sampling on the last stage feature in the RegNet lightweight network and performing feature fusion with the last but one stage feature;

and S23, inputting the image after feature fusion into a single-scale YOLO layer for instrument target recognition and classification, and obtaining a classified instrument image.

In the step S21, the RegNet lightweight network search follows the idea of combining the NAS search result and the manual design, combines the advantages of the two, and aims to reduce the search space of the NAS by using the manual design idea. As shown in fig. 3, the RegNet lightweight network includes a convolutional layer, a first RegBlock unit, a second RegBlock unit, a third RegBlock unit, and a fourth RegBlock unit, which are connected in sequence;

the first RegBlock unit, the second RegBlock unit, the third RegBlock unit and the fourth RegBlock unit sequentially

comprise

1, 4 and 7 RegBlock layers which are sequentially connected, and the number of channels of the RegBlock layers is 24, 56, 152 and 368;

each RegBlock layer comprises a first 1 × 1 convolutional layer, a first 3 × 3 convolutional layer, a second 1 × 1 convolutional layer and a feature fusion layer which are connected in sequence, and the input end of the first 1 × 1 convolutional layer is also connected with the feature fusion layer through a down-sampling layer;

in step S22, the feature output by the fourth RegBlock unit is used as the last stage feature, and the feature output by the third RegBlock unit is used as the second-to-last stage feature.

The network parameters of the RegNet lightweight network are shown in table 1;

table 1: RegNet lightweight network structure

In step S22, the convolution features extracted by RegNet are not directly input into the YOLO layer for prediction, because the shallow layer of the convolutional neural network contains more detailed information, and the deep layer contains more semantic information. If the output characteristics of the last layer are directly input into the YOLO layer for prediction, the abundant detail information of the shallow layer can be lost, so that in the step, the characteristic fusion is carried out in a characteristic splicing mode, the shallow layer characteristics are reserved, and the fusion of the characteristics of different scales is also beneficial to the network to identify the targets of different scales.

In step S23, the size scale of the meter does not vary much for meter identification, and the ratio of the meter in the image is generally neither too large nor too small for ensuring the accuracy of subsequent meter readings. Therefore, based on the specificity of this instrument recognition task, it is proposed to use single-scale YOLO for instrument recognition and only use intermediate-scale features for prediction. On the one hand, the complexity of the algorithm can be reduced, the operation efficiency of the algorithm is improved, and on the other hand, for the task that the scale of the instrument identification is not changed much, the scale is reduced, and the detection accuracy of the instrument cannot be reduced.

In step S2 of this embodiment, the loss function in the process of identifying and classifying the instrument target includes the positioning loss l_boxClassification loss l_clsAnd confidence loss l_noobj；

Wherein, the positioning loss l_boxThe method is used for correcting the predicted accurate position coordinates and comprises the following calculation formula:

in the formula, λ_coordIs the coordinate constant parameter of the bounding box, is the size of the original image divided into cells, B is the bounding box,

a control variable representing the existence of an instrument target in the original image at the (i, j) position, wherein the existence of the instrument target is 1, and otherwise, the existence of the instrument target is 0, w_iTo enclose the frame width, h_iTo the height of the bounding box, x_iTo predict the center point abscissa of the resulting instrument target,

is the center point abscissa, y, of the real instrument target_iTo predict the center point ordinate of the resulting instrument target,

is the ordinate of the center point, w, of the real instrument target_iTo predict the resulting instrument target bounding box width,

width of bounding box for real instrument target, h_iTo predict the resulting meter target bounding box height,

the bounding box height for the real instrument target;

classification loss l in this example_clsMake it impossibleThe traditional SoftMax cross entropy loss is used for calculating the multi-classification loss, because the same object can have a plurality of marks, and the use of the SoftMax loss can cause the network to default to only one mark for each target, thereby causing the generation of errors; therefore, in this step, the classification loss is replaced by a binary cross entropy loss, which is obtained by traversing each class and is calculated by the following formula:

where classes is a set of instrument target classifications, P_i ^jTo predict the probability of a meter target belonging to a class, P_i ^％Is the probability that the meter target actually belongs to a certain class;

loss of confidence l_noobjThe method is used for supervising the network to find a correct target in the image and giving the confidence level of the object contained in the current bounding box, and the calculation formula is as follows:

in the formula (I), the compound is shown in the specification,

to calculate the confidence by cross-comparing the predicted bounding box with the true label box,

for the purpose of the confidence level,

and if the control variable of the object exists in the current surrounding frame, the control variable is 1 if the object exists, and otherwise, the control variable is 0. It can be seen that the loss is composed of two parts, the first part represents the contribution to the loss when there is a current target, and the other part represents the contribution to the loss when there is no current target, and then a coefficient is multiplied to weaken the contribution to the loss, because of the target detection taskThe number of backgrounds is much larger than the foreground.

In step S2 of the present embodiment, the instrument target categories mainly include a square pointer instrument, a round pointer instrument, and a digital instrument.

In step S3 of this embodiment, a fine-grained classification network combined with attention-blending and clipping is proposed, in which a Weakly Supervised attention Data enhancement network (WS-DAN, weak Supervised Data augmentation network) is improved, and the learning of a local area by the enhancement network is performed by using an attention mechanism to guide the improved blending and clipping Data enhancement. Based on this, step S3 specifically includes:

s31, extracting the features of the current input image through the feature extraction network of the fusion channel attention module;

s32, processing the extracted features through M1 × 1 convolution layers to obtain M attention maps with position information;

s33, fusing the attention diagrams and the extracted feature diagrams by utilizing bilinear attention pooling, and calculating cross entropy loss through a full connection layer to obtain a plurality of characteristic matrixes corresponding to the attention diagrams;

s34, based on the position information in the attention map, performing mixed cropping and data enhancement on the image, and inputting the image into the feature extraction network of the fusion channel attention module in the step S31;

and S35, determining the measuring range information of the instrument image based on the currently obtained characteristic matrix.

The step S32 is specifically: extracting a feature map F epsilon R of the input image I through a convolutional neural network^H*W*CWhere H, W represents the length and width of the feature map, C represents the number of channels of the feature map, and F is then converted into an attention map A ∈ R by M1 × 1 convolution kernels^H*W*MThe value of M is a hyperparameter and represents the number of the attention diagrams, and the calculation formula of the attention diagram A is as follows:

wherein f (F) is a1 × 1 convolution operation;

the step S33 is specifically: for each attention map A_kMultiplying the element by element to the original characteristic diagram F to obtain M bilinear characteristic diagrams F epsilon R of the strengthened local characteristics^1*NAnd the effect of enhancing fine-grained identification is achieved. Meanwhile, in order to reduce the feature dimension, M f are subjected to global average pooling or global maximum pooling_kAnd performing discriminant local feature extraction to obtain M local feature vectors. Finally, the local features are spliced to obtain a final feature matrix, and the obtained feature matrix is as follows:

wherein P represents the final feature matrix P ∈ RM x N resulting from stitching, the symbol indicates element-by-element product, g (·) indicates global pooling, and Γ (A, F) indicates bilinear pooling for the original feature map F of the attention map A. The bilinear feature obtained by the method can obtain the second-order feature representation of the image, and meanwhile, the 1 x 1 convolution is utilized to carry out channel aggregation on the high-dimensional feature image, so that the dimension of the feature is reduced, and the calculated amount is reduced.

In step S34, the method for performing blend cropping and data enhancement on the image specifically includes:

for the respective attention diagram A_kFirstly, regularization is carried out to enhance the response contrast of the region and the region is converted into a characteristic thermodynamic diagram

In the formula, for characteristic thermodynamic diagram

Averaging 32 channels to obtain the position information with the strongest response to the picture, and then calculating according to a given threshold value thetaMask M for cutting out region_ciIn particular for characteristic thermodynamic diagrams

If it is greater than theta, mask M_ciThe corresponding position is 1, otherwise 0. The calculation formula is as follows:

in the formula, (m, n) represents a characteristic thermodynamic diagram or an abscissa and ordinate value of the mask.

May then follow mask M_ciObtaining a surrounding frame Bi capable of surrounding all the areas larger than the threshold value, and cutting out the target picture x from the original image according to the coordinates of the surrounding frame_c1、x_c2. Then x is put into_c1Is adjusted to x_c2Is obtained by

x_c2Is adjusted to x_c1Is obtained by

Finally respectively make

Is filled in to x₂B of (A)₂Position of will

Is filled in to x₁B of (A)₁And (3) completing mixed cutting of the image by using the position, wherein the calculation formula is as follows:

wherein R (A, B) represents the adjustment of the A picture to the size of the B picture (mask),

and

respectively represent by x₁,x₂And obtaining a new picture.

Correspondingly, the method is different from the original mixed cutting data enhancement in that the weighted sum is calculated according to the area ratio, the most main and most discriminant characteristics of the pictures can be completely cut out and exchanged according to the attention-guided mixed cutting, and therefore the real marks y of the two pictures are marked₁，y₂The exchange is carried out as follows:

in the formula (I), the compound is shown in the specification,

and

respectively correspond to

And

the image mark of (1).

Then will obtain

The network is sent again for training, the learning of the network to the local area is enhanced, and the network pair is reducedThe likelihood of overfitting the environment.

In step S4 of this embodiment, after the specific position of the meter and the meter range information in the image are acquired, the meter reading needs to be performed on the identified different meters by using a targeted algorithm, and this embodiment mainly includes performing meter reading on square pointer meters, circular pointer meters, and digital meters.

Specifically, when the target of the meter is a square pointer meter, the method of reading the meter is shown in fig. 4, specifically:

a1, performing Gaussian blur and image graying processing on the instrument image only with the instrument target to obtain a grayscale image;

a3, detecting the pointer in the edge binary image by a straight line detection method, and correcting the pointer angle to obtain the pointer angle;

and A4, calculating the reading of the meter according to the measuring range information and the pointer angle of the meter image to obtain the reading value of the meter.

In the step a1, the instrument image is blurred by gaussian filtering to avoid the influence of noise points in the image, where the gaussian filtering is a smooth linear filter, and a template is used to scan all pixels in the image in a sliding window manner, and the template has a weighted value with gaussian distribution, and the final value of the element at the center point of the template is obtained after weighted averaging is performed on the pixels in the template according to the weighted value.

When the image is subjected to gray processing, the gray processing based on the red channel is performed by adopting the color characteristic that the pointer to be identified is red, and the red channel component is taken as a final gray value. Compared with the standard gray image, the gray image obtained in the way has more obvious characteristics of the pointer in the image, and is more beneficial to the subsequent steps.

In the step a2, after the gray-scale image is obtained, edge detection needs to be performed on the image, and the Canny edge detection algorithm is adopted in the embodiment, which has the advantages of low error rate, accurate edge point positioning, and no repeated response.

In the step a3, pointer detection is performed by a line detection method, in this embodiment, line detection is performed by hough transform, and the hough transform transforms the middle of the image into a parameter space, so that the line detection problem is transformed into the parameter space to obtain a peak value. Specifically, when a plurality of straight lines are detected in the image by hough straight line detection, the straight lines are further filtered by using conditional constraints, wherein the straight lines include edge straight lines of a meter and a pointer of the meter. Wherein, the constraint conditions of the edge straight line of the instrument comprise: 1) the length is long enough: in the instrument local picture obtained by target detection, the edge should be comparable to the width of the image, so that among all the lines detected, the edge line should be long enough; 2) at the edge of the image: the instrument image has been cropped for target detection, so the edge of the instrument should appear at the edge of the image; 3) the slope should be within ± 0.5 and the detected edge line should be nearly horizontal in order to prevent confusion with the pointer line. For the pointer of the meter, in order to take the final reading, it is necessary to detect the pointer in the image, and then calculate the final reading by the deflection angle of the pointer. The constraint on the pointer also takes three constraints: first, the deflection angle of the pointer should be in the range of 45 ° to 135 °, and a certain error range is set in order to exclude the deflection of the photographed picture, so the slope to the straight line of the pointer should be greater than 0.8 or less than-0.8. The next is that the pointer has its own center of deflection, which is set to the center of the image down 1/4 depending on the meter characteristics. Therefore, the straight line where the pointer is located should be the closest to the point among all the straight lines, so filtering of the straight lines is performed by calculating the distance from the point to the straight line.

In the step a4, two straight lines are obtained through the above steps, and then the slopes k1 and k2 are calculated, and then two angle information is calculated through an inverse trigonometric function, one of which is the horizontal inclination angle θ of the image₁The second is the inclination angle theta of the pointer₂(ii) a Further, the calculation formula for obtaining the meter reading value result is as follows:

When the target of the meter is a circular pointer meter, the reading method of the meter is shown in fig. 5, and specifically comprises the following steps:

b1, performing Gaussian blur and image graying processing on the instrument image only with the instrument target to obtain a grayscale image;

b4, straightening the circular scales by coordinate transformation on the instrument target in the outer contour;

b5, determining the scale position after straightening treatment in a horizontal projection mode, and determining the meter reading according to the length of the scale position;

the gaussian processing and the image graying processing in step B1 are performed in the same manner as in the case of the square pointer instrument.

The local adaptive threshold binarization in the step B2 is to determine a threshold of a binarization result of a certain pixel in the image, specifically, the threshold of the binarization result is determined by all pixels in a neighborhood of a fixed size, and a value of the adaptive threshold may be calculated by average gray scale or gaussian weighting.

In the step B3, the principle of circle detection by hough transform is similar to that of straight line detection. Three points may define a unique circle, which also maps the circle in image space into parameter space. For the circular equation in image space:

after a, b and r are regarded as variables, the equation is converted into a representation of a parameter space, and the geometric meaning of the representation is a curve in a three-dimensional space. That is, all circles in the image space passing a certain point are converted into a curve in the three-dimensional parameter space. And then the peak value of the curve intersection point can be calculated through a voting algorithm, and then the equation of the potential circle is solved. And because the peak value statistics in the three-dimensional space is high in time consumption, in order to improve the calculation speed as much as possible, the fast Hough transform is adopted. According to the method, a three-dimensional Hough space is converted into a two-dimensional Hough space, so that the operation rate is increased by one order of magnitude, and the Hough circle detection efficiency is greatly improved. Then, for a plurality of circular equations searched in the instrument image, the best fitting one to the instrument outline is filtered out in a condition constraint mode. The specific constraint conditions are as follows: the diameter is the largest among all circles with their centers in the middle of the image. The reason is as follows: for an instrument image obtained by instrument detection, the center of the image is closer to the center of the instrument; secondly, its diameter should also be comparable to the image width. The best fitting circle to the meter profile can be filtered out according to the two constraints mentioned above.

In step B4, the instrument in cartesian coordinate system is converted to polar coordinate system, so that the square instrument panel can be straightened, i.e. the angle problem is converted to length problem.

In the step B5, for the pointer location of the binary image, the horizontal projection method is used to find the peak value of the horizontal projection of the black pixel in the binary image, because the conversion center of the polar coordinate system is converted to be the center of the meter and also the rotation center of the pointer, the pointer after conversion should be a horizontal line, and the peak value of the black pixel is obtained after the horizontal projection, that is, the position of the pointer can be confirmed. The horizontal projection means that for a binary image, the number of black pixels in each line is counted to obtain an array with the size of the image height, and the corresponding position of the array is the number of black pixels in a certain line in the image.

Similarly, the position of the zero scale mark can be found by utilizing the characteristics of the projection drawing, so that the length d from the zero scale mark to the position of the pointer can be calculated₁According to the prior knowledge, the total length d of the scale mark can be obtained₂. Therefore, the calculation formula of the obtained meter reading value result is as follows:

When the target of the meter is a digital meter, the method of reading the meter is shown in fig. 6, specifically:

c1, performing Gaussian blur on the instrument image only with the instrument target, and extracting the digital part in the image by adopting a red channel-based binarization method;

and C4, identifying the segmented numbers by using a Lenent-5 network, and completing the reading of the meter.

In step C1, the gaussian blur processing method is the same as that of the first two meters.

In the step C3, the digital segmentation by the vertical projection method means that there is a feature in the binarized image, in which the number of target color pixels of the target object in the vertical or horizontal direction is one, and the horizontal and vertical coordinates of the object can be known by using the feature, and the object can be further segmented from the original image.

When characters needing to be segmented exist in the digital display area, the characters are identified and segmented by adopting a thinning algorithm; specifically, a certain gap is ensured to exist between characters through a thinning algorithm so as to prevent the characters from being cut into a plurality of characters, a classic rapid parallel thinning algorithm is used in the embodiment, efficiency and accuracy are ensured, the algorithm is divided into two stages, all foreground pixel points are scanned in the first stage, and the pixel points meeting three conditions are marked as deletion points. Firstly, the number of the 0 pixel points in the eight-neighborhood is between two and six, secondly, the change times of the sequential pixels in the eight-neighborhood is 1, and then at least one of the four neighborhoods in the upper-right-lower direction or the left-right-lower direction is 0. The second stage is the same as the first two steps in the first stage, except that in the third step, at least one pixel in the upper-right-lower or left-right-lower direction of the four neighborhood is changed to 0 in the upper-left-lower or upper-left-lower direction of the four neighborhood. The two steps are cycled until no pixels can be deleted, i.e. the refinement process is completed.

In the step C4, Lenet-5 is composed of two convolutional layers, two downsampling layers, and three full-connection layers with different connection modes, is a classic network for identifying handwritten numbers, and has the characteristics of low calculation amount, simple structure and high robustness.

Example 2:

in this embodiment, the operating efficiency of the meter identification and reading method is tested and analyzed on an intel (r) core (tm) i7-8700 CPU, and the test indexes include time consumption, total time consumption, and reading accuracy of each stage. The reading precision refers to that after the reading is manually judged, if the reading given by the meter recognition system is within the range of the minimum unit scale, the reading is regarded as a correct reading, and otherwise, the reading is regarded as an incorrect reading. The ratio of correct identification to total identification was calculated and recorded as reading accuracy and is detailed in table 2.

Table 2: meter identification and reading system performance analysis

The method for identifying and reading the instrument can finish the detection task of the instrument in a short time, and can finish the instrument detection within 70ms even if the GPU is not used for acceleration. And the fine-grained identification range can be completed in about 110 ms. The reading time of the instrument is greatly different according to different reading modes of different types of instruments, and the square pointer instrument does not use an algorithm with large calculation amount, so that the reading time of the instrument is low, and the identification rate of 5FPS (Frame per Second) can be approached. And the round pointer instrument uses the time-consuming operation of Hough circle detection, so that the total time consumption is close to 1 second. And for a digital instrument, fine-grained range identification operation is not required, but each cut nixie tube number needs to pass through the Lenet-5 in series, so that the identification rate is equivalent to that of a square pointer and can approach the processing rate of 5 FPS. In terms of reading accuracy, the recognition accuracy of the digital instrument is the highest and reaches 98%, while the reading accuracy of the square pointer is 92% due to shaking caused by hand-held shooting or angle deformation caused by shooting angles and the like, and the reading accuracy of the circular pointer reaches 94%.

Generally speaking, the system achieves the processing speed close to real time in the CPU environment, and meanwhile, the recognition precision is high, and the requirement in actual production can be met. Meanwhile, due to the low calculation amount, the method can be transferred to various practical scenes with limited calculation capacity, and has wide application prospect.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A meter identification method based on a lightweight network is characterized by comprising the following specific steps:

acquiring an original image with a meter;

fusing the features in the first extracted feature map to obtain a fused image;

2. The instrument recognition method based on the lightweight network according to claim 1, wherein the step of obtaining the fusion image is as follows: and performing double up-sampling on the N stage characteristics in the RegNet lightweight network, and performing characteristic fusion on the N stage characteristics and the N-1 stage characteristics, wherein the RegNet lightweight network comprises N stages.

3. The meter identification method based on the lightweight network as claimed in claim 1 or 2, wherein the meter target classification comprises square pointer meter, round pointer meter and digital meter.

4. The method of claim 3, wherein the loss function in the process of identifying and classifying the targets of the meters comprises the positioning loss l_boxClass loss l_clsAnd confidence loss l_noobj。

5. A meter reading method based on a lightweight network is characterized by comprising the following specific steps:

acquiring an image of the classification instrument;

based on the range information of the meter image, a meter reading is obtained.

6. The method for reading the meter based on the lightweight network according to claim 5, wherein the specific steps for acquiring the range information of the meter image are as follows:

s24, based on the position information in the attention diagrams, performing mixed clipping and data enhancement on the attention diagrams, and inputting the result into the feature extraction network of the fusion channel attention module in the step S21;

7. The method for reading the meter based on the lightweight network as claimed in claim 5 or 6, wherein when the meter target is a square pointer meter, the method for reading the meter is specifically as follows:

8. The method for reading the meter based on the lightweight network as claimed in claim 5 or 6, wherein when the meter target is a circular pointer meter, the method for reading the meter is specifically as follows:

b4, straightening the circular scales through coordinate transformation;

9. The method for reading the meter based on the lightweight network according to claim 5 or 6, wherein when the meter target is a digital meter, the method for reading the meter is specifically as follows:

and C4, utilizing a Lenent-5 network to identify the segmented numbers.

10. The meter reading method based on the lightweight network as claimed in claim 9, wherein in step C3, when there is a character to be segmented in the digital display area, a refinement algorithm is used to identify and segment the character.