CN112308054B - Automatic reading method of multifunctional digital meter based on target detection algorithm - Google Patents
Automatic reading method of multifunctional digital meter based on target detection algorithm Download PDFInfo
- Publication number
- CN112308054B CN112308054B CN202011585525.8A CN202011585525A CN112308054B CN 112308054 B CN112308054 B CN 112308054B CN 202011585525 A CN202011585525 A CN 202011585525A CN 112308054 B CN112308054 B CN 112308054B
- Authority
- CN
- China
- Prior art keywords
- yolov4
- digital
- multifunctional digital
- switch cabinet
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a target detection algorithm-based automatic reading method for a multifunctional digital meter, which comprises the following steps of: step S1, carrying out lightweight improvement on the YOLOv4 network model to obtain a YOLOv4 lightweight network model; step S2, collecting the picture of the multifunctional digital meter and calibrating the position and the category of the digit on the picture of the multifunctional digital meter; and step S3, training the YOLOv4 lightweight network model by using the multifunctional digital chart picture calibrated in the step S2 and the calibration information thereof, and obtaining the YOLOv4 lightweight digital detection model. The multifunctional digital meter automatic reading method based on the target detection algorithm has the advantages of high identification speed, high identification accuracy and good stability, and solves the problems that the existing digital meter automatic reading method is easily influenced by external environment change and good and bad image imaging quality and lacks digital identification stability and accuracy.
Description
Technical Field
The invention relates to the technical field of reading of multifunctional digital meters, in particular to an automatic reading method of a multifunctional digital meter based on a target detection algorithm.
Background
The multifunctional digital meter has the advantages of being visual in display, multifunctional, high in cost performance and the like, is widely used in a power distribution room, can realize display of different unit currents and voltages through simple setting, is generally displayed in a three-row and four-column mode, namely three readings can be displayed simultaneously, each reading is formed by assembling four numbers and decimal points, and each number and decimal point are represented by a red light emitting diode (as shown in figure 1). In the daily inspection of the power distribution room, professional workers are required to watch and record the reading of the digital meter through human eyes, the inspection efficiency is low, and the reading is easy to make mistakes.
The existing automatic reading method of the digital meter is basically realized in two steps, namely, in the first step, the position of the whole digital meter or the position of a digital display area in a dial is determined by a target detection algorithm (such as fast-rcnn, SSD or YOLO); and the second step is combined with a basic image processing algorithm to realize single digit extraction and segmentation on the basis of the first step, and finally, the digit recognition is realized through template matching or a classifier, or the digit recognition is realized through a text recognition method on the basis of the first step. In the second step, the methods all use the traditional image algorithm to realize digital identification, are easily influenced by external environment change and image imaging quality, lack stability and accuracy of digital identification, and cannot meet the requirements of motion characteristics and identification precision of the inspection robot.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a multifunctional digital meter automatic reading method based on a target detection algorithm, which has the advantages of high recognition speed, high recognition accuracy and good stability, and solves the problems that the existing digital meter automatic reading method is easily influenced by external environment change and good and bad image imaging quality and lacks digital recognition stability and accuracy.
In order to achieve the purpose, the invention adopts the following technical scheme:
a multifunctional digital meter automatic reading method based on a target detection algorithm comprises the following steps:
step S1, carrying out lightweight improvement on the YOLOv4 network model to obtain a YOLOv4 lightweight network model;
step S2, collecting the picture of the multifunctional digital meter and calibrating the position and the category of the digit on the picture of the multifunctional digital meter;
step S3, training a YOLOv4 lightweight network model by using the multifunctional digital chart picture calibrated in the step S2 and the calibration information thereof to obtain a YOLOv4 lightweight digital detection model;
step S4, carrying out digital detection and identification on the multifunctional digital table picture by using the YOLOv4 lightweight digital detection model obtained in the step S3, and returning the information of each detected number;
the multifunctional digital meter picture is obtained by detecting a picture shot by the inspection robot through a YOLOv4 switch cabinet device detection model;
step S5, according to the returned information of each number, each number is sequenced and recombined to obtain a correct number arrangement sequence and obtain a correct reading;
and step S6, adding decimal points to the reading according to the preset decimal number to obtain the final reading.
To be further described, the YOLOv4 switch cabinet device detection model is a trained detection model, and the training of the YOLOv4 switch cabinet device detection model includes the following steps:
(1) establishing a training set: manually marking a plurality of switch cabinet pictures shot by the inspection robot in a multi-time inspection mode by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing a training set;
(2) establishing a test set: manually marking a plurality of switch cabinet pictures which do not participate in training by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing to obtain a test set;
(3) training an original YOLOv4 target detection network model on a training set, and obtaining a YOLOv4 switch cabinet device detection model after iterative training;
(4) the detection performance of the YOLOv4 switch cabinet device detection model is tested on a test set.
Further, the step S1 specifically includes the following steps:
step S10, replacing a CSPDarknet53 network structure in the original YOLOv4 network model with a lightweight network structure MobileNet V3 to serve as a backbone network;
step S11, deleting the two posterior YOLOv3head structures in the original YOLOv4 network model, only reserving one YOLOv3head structure, and reducing the number of anchor points;
in step S12, the net input is reduced from 416 by 416 to 192 by 256.
Further, the step S2 specifically includes the following steps:
s20, screening out pictures containing the multifunctional digital meter from a large number of switch cabinet pictures shot by the inspection robot in the inspection process, and manually framing out the position of the multifunctional digital meter area;
step S21, cutting out a single multifunctional digital table picture from the picture of the switch cabinet according to the position rectangular frame selected by the manual frame and storing the single multifunctional digital table picture;
and step S22, performing position calibration and category calibration on the stored numbers on each multifunctional digital table picture by using the labeling software, and storing the labeling information in an XML file.
Further, the step S3 is specifically: and (4) training the YOLOv4 lightweight network model by using the multifunctional digital table pictures calibrated in the step S2 and the corresponding XML files, and performing iterative training to obtain a YOLOv4 lightweight digital detection model.
Further, the step S4 specifically includes the following steps:
step S40, detecting pictures shot by the inspection robot by using a YOLOv4 switch cabinet device detection model to obtain a multifunctional digital table, and cutting the pictures from the switch cabinet pictures according to a detection rectangular frame to obtain multifunctional digital table pictures;
and step S41, inputting the cut multifunctional digital chart picture into a YOLOv4 lightweight digital detection model for digital detection, and returning position rectangular frame information and category information of each number.
Further, the step S5 specifically includes the following steps:
step S50, returning position rectangular frame information and category information of each number according to a YOLOv4 lightweight number detection model, sorting each number according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, and dividing the identified number into three rows, namely an upper row, a middle row and a lower row;
and step S51, sequencing the digital detection results belonging to the same row from left to right according to the size of the abscissa of the left upper vertex of the rectangular frame at the position of the digital detection results, finishing sequencing recombination, obtaining the correct digital arrangement sequence, and obtaining the correct reading.
Further, the step S6 specifically includes the following steps:
step S60, before the inspection robot formally inspects the switch cabinet of the power distribution room, the staff plan the stepping points of the robot motion path through the Qt upper computer interface controlling the robot motion;
and step S61, when the inspection robot moves to step on the point, if the shooting point has a multifunctional digital table, inputting the decimal point digit in the decimal point adding column of the Qt interface.
Compared with the prior art, the invention has the following beneficial effects:
1. by carrying out lightweight improvement on the original YOLOv4 network model, the hardware running cost of the algorithm is reduced, the digital detection recognition speed is improved, and the recognition time of each digital table is only 6.2 ms. Based on Darknet frame training, the method can conveniently support different system deployments of Windows and Linux, and has the characteristics of high recognition speed and convenience in deployment;
2. the position and the recognition result of each digit in the digital table can be obtained by using the deep learning target detection algorithm with strong learning capacity, the realization is simple, the method is not easily influenced by external environment change and image imaging quality, the average digit recognition accuracy rate under various different environments is 99.7 percent, the method has higher recognition accuracy rate and good stability, and the method is applied to a power distribution room inspection robot system and has the characteristics of high efficiency, accuracy and stability;
3. the multifunctional digital meter area is selected through a manual frame, then the position calibration and the category calibration are carried out on the stored numbers on each multifunctional digital meter picture, the marking accuracy rate is high, the accuracy rate of training the YOLOv4 lightweight network model is ensured, and the identification accuracy rate of the YOLOv4 lightweight digital detection model is ensured.
Drawings
The drawings are further illustrative of the invention and the content of the drawings does not constitute any limitation of the invention.
FIG. 1 is a schematic diagram of a multi-purpose digital watch of one embodiment of the present invention;
FIG. 2 is a flow chart of a method for automatic reading of a multi-function digital meter based on a target detection algorithm according to one embodiment of the present invention;
FIG. 3 is a schematic illustration of a picture of a switchgear cabinet according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a portion of a device cut out from a picture of a switchgear according to an embodiment of the present invention, wherein (a) is a temperature and humidity controller, (b) is a voltage converting switch, (c) is a multifunctional digital meter, (d) is a trip tab, (e) is a live indication, and (f) is a ground fault indication;
FIG. 5 is a schematic diagram of the raw test results obtained from the YOLOv4 lightweight digital test model in accordance with one embodiment of the present invention;
FIG. 6 is a graphical representation of the results of sequenced reads after sequencing reorganization in accordance with one embodiment of the present invention;
FIG. 7 is a diagram illustrating sorted and reorganized rectangular box inner number recognition results according to an embodiment of the present invention.
Detailed Description
A multifunctional digital meter automatic reading method based on a target detection algorithm comprises the following steps:
step S1, carrying out lightweight improvement on the YOLOv4 network model to obtain a YOLOv4 lightweight network model;
step S2, collecting the picture of the multifunctional digital meter and calibrating the position and the category of the digit on the picture of the multifunctional digital meter;
step S3, training a YOLOv4 lightweight network model by using the multifunctional digital chart picture calibrated in the step S2 and the calibration information thereof to obtain a YOLOv4 lightweight digital detection model;
step S4, carrying out digital detection and identification on the multifunctional digital table picture by using the YOLOv4 lightweight digital detection model obtained in the step S3, and returning the information of each detected number;
the multifunctional digital meter picture is obtained by detecting a picture shot by the inspection robot through a YOLOv4 switch cabinet device detection model;
step S5, according to the returned information of each number, each number is sequenced and recombined to obtain a correct number arrangement sequence and obtain a correct reading;
and step S6, adding decimal points to the reading according to the preset decimal number to obtain the final reading.
Specifically, the automatic reading method of the multifunctional digital meter based on the target detection algorithm is applied to a power distribution room inspection robot system.
The method specifically comprises the steps of realizing automatic reading of the multifunctional digital meter in two steps, namely detecting a cabinet surface picture shot by an inspection robot by using a YOLOv4 switch cabinet device detection model to obtain a multifunctional digital meter picture, and then directly realizing detection and identification of each number in the digital meter by using a YOLOv4 light-weight digital detection model improved in light weight;
according to the method, the original Yolov4 network model is improved in light weight, the hardware operation cost of the algorithm is reduced, the digital detection recognition speed is improved, and the recognition time of each digital table is only 6.2 ms. Based on Darknet frame training, the method can conveniently support different system deployments of Windows and Linux, and has the characteristics of high recognition speed and convenience in deployment;
in addition, the position and the recognition result of each digit in the digital table can be obtained by using the deep learning target detection algorithm with strong learning capacity, the realization is simple, the influence of external environment change and image imaging quality is not easy to occur, the average digital recognition accuracy under various different environments is 99.7 percent, the high recognition accuracy is realized, the stability is good, and the method is applied to a power distribution room inspection robot system and has the characteristics of high efficiency, accuracy and stability.
To be further described, the YOLOv4 switch cabinet device detection model is a trained detection model, and the training of the YOLOv4 switch cabinet device detection model includes the following steps:
(1) establishing a training set: manually marking a plurality of switch cabinet pictures shot by the inspection robot in a multi-time inspection mode by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing a training set;
(2) establishing a test set: manually marking a plurality of switch cabinet pictures which do not participate in training by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing to obtain a test set;
(3) training an original YOLOv4 target detection network model on a training set, and obtaining a YOLOv4 switch cabinet device detection model after iterative training;
(4) the detection performance of the YOLOv4 switch cabinet device detection model is tested on a test set.
As the inspection robot has various instrument devices in switch cabinet pictures shot at different inspection points, as shown in figure 3, besides multifunctional numbers, the switch cabinet pictures also have devices such as an indicator light, a large switch, a temperature and humidity controller and the like, the method firstly trains an original YOLOv4 target detection network model to obtain a YOLOv4 switch cabinet device detection model, then detects the multifunctional digital tables and other machines on the switch cabinet pictures by using a YOLOv4 switch cabinet device detection model (Object _ Detector) to obtain the position frame of each device and the type of the device, then cuts out the device from the switch cabinet pictures according to the position frame of each device as shown in figure 4, calls the corresponding interface function according to the type of the device to identify the working state, detects to obtain the multifunctional digital table pictures, and has high detection accuracy and high detection efficiency.
Further, the step S1 specifically includes the following steps:
step S10, replacing a CSPDarknet53 network structure in the original YOLOv4 network model with a lightweight network structure MobileNet V3 to serve as a backbone network;
step S11, deleting the two posterior YOLOv3head structures in the original YOLOv4 network model, only reserving one YOLOv3head structure, and reducing the number of anchor points;
in step S12, the net input is reduced from 416 by 416 to 192 by 256.
Through carrying out the lightweight improvement to former YOLOv4 network model, can effectively reduce the model calculation volume, reduce the hardware running cost of algorithm, when guaranteeing digital detection discernment rate of accuracy, can effectively promote digital detection discernment speed to improve the work efficiency who patrols and examines the robot.
Further, the step S2 specifically includes the following steps:
s20, screening out pictures containing the multifunctional digital meter from a large number of switch cabinet pictures shot by the inspection robot in the inspection process, and manually framing out the position of the multifunctional digital meter area;
step S21, cutting out a single multifunctional digital table picture from the picture of the switch cabinet according to the position rectangular frame selected by the manual frame and storing the single multifunctional digital table picture;
and step S22, performing position calibration and category calibration on the stored numbers on each multifunctional digital table picture by using the labeling software, and storing the labeling information in an XML file.
The multifunctional digital meter area is selected through a manual frame, then the position calibration and the category calibration are carried out on the stored numbers on each multifunctional digital meter picture, the marking accuracy rate is high, the accuracy rate of training the YOLOv4 lightweight network model is ensured, and the identification accuracy rate of the YOLOv4 lightweight digital detection model is ensured.
Further, the step S3 is specifically: and (4) training the YOLOv4 lightweight network model by using the multifunctional digital table pictures calibrated in the step S2 and the corresponding XML files, and performing iterative training to obtain a YOLOv4 lightweight digital detection model.
The YOLOv4 lightweight network model is trained by using the multifunctional digital table pictures calibrated in the step S2 and the corresponding XML files, so that the obtained YOLOv4 lightweight digital detection model can identify the position and the type of each number in the multifunctional digital table, and is high in identification accuracy, good in stability and not easily influenced by external environment changes and image imaging quality.
Further, the step S4 specifically includes the following steps:
step S40, detecting pictures shot by the inspection robot by using a YOLOv4 switch cabinet device detection model to obtain a multifunctional digital table, and cutting the pictures from the switch cabinet pictures according to a detection rectangular frame to obtain multifunctional digital table pictures;
and step S41, inputting the cut multifunctional digital chart picture into a YOLOv4 lightweight digital detection model for digital detection, and returning position rectangular frame information and category information of each number.
The cut multifunctional digital table picture is input into a YOLOv4 lightweight digital detection model for digital detection, accuracy of position rectangular frame information and category information of each digital detected and returned is high, sequencing and recombination can be conveniently carried out according to results of the rectangular frame information, and sequencing accuracy is high.
Further, the step S5 specifically includes the following steps:
step S50, returning position rectangular frame information and category information of each number according to a YOLOv4 lightweight number detection model, sorting each number according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, and dividing the identified number into three rows, namely an upper row, a middle row and a lower row;
and step S51, sequencing the digital detection results belonging to the same row from left to right according to the size of the abscissa of the left upper vertex of the rectangular frame at the position of the digital detection results, finishing sequencing recombination, obtaining the correct digital arrangement sequence, and obtaining the correct reading.
And sequencing each number according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, sequencing the detection results of the numbers belonging to the same row from left to right according to the size of the horizontal coordinate of the top left vertex of the position rectangular frame, finishing sequencing recombination, wherein the sequencing recombination method is simple and the sequencing efficiency is high.
Further, the step S6 specifically includes the following steps:
step S60, before the inspection robot formally inspects the switch cabinet of the power distribution room, the staff plan the stepping points of the robot motion path through the Qt upper computer interface controlling the robot motion;
and step S61, when the inspection robot moves to step on the point, if the shooting point has a multifunctional digital table, inputting the decimal point digit in the decimal point adding column of the Qt interface.
The readings obtained after the sequencing and the recombination in the step S5 do not contain decimal points, and because the camera of the robot is far away from the switch cabinet, and the movement characteristic of the robot and the display characteristic of the multifunctional digital meter, the decimal point position of each row of readings in the multifunctional digital meter is difficult to determine through an image processing algorithm, so that the accuracy and the stability are very low, and the great difference of ten times or thousand times exists in the readings.
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
As shown in fig. 2, a method for automatically reading a multifunctional digital meter based on a target detection algorithm comprises the following steps:
step S1, carrying out lightweight improvement on the YOLOv4 network model to obtain a YOLOv4 lightweight network model, and specifically comprising the following steps:
step S10, replacing a CSPDarknet53 network structure in the original YOLOv4 network model with a lightweight network structure MobileNet V3 to serve as a backbone network;
step S11, deleting the two subsequent YOLOv3head structures in the original YOLOv4 network model, only reserving one YOLOv3head structure, and reducing the anchor point number (anchors) from 18 to 10;
in step S12, the net input is reduced from 416 by 416 to 192 by 256.
Step S2, collecting the pictures of the multifunctional digital meter, and performing position calibration and category calibration on the numbers on the pictures of the multifunctional digital meter, specifically comprising the following steps:
s20, screening out pictures containing the multifunctional digital meter from a large number of switch cabinet pictures shot by the inspection robot in the inspection process, and manually framing out the position of the multifunctional digital meter area;
step S21, cutting 6000 single digital table pictures of the multifunctional table from the picture of the switch cabinet according to the position rectangular frame selected by the manual frame, and storing the pictures in a local disk;
and step S22, performing position calibration and category calibration on the stored numbers on each multifunctional digital table picture by using labeling software (specifically, Labelimage labeling software), and storing the labeling information in an XML file.
Step S3, using 6000 sheets of multifunctional digital table pictures and corresponding XML files calibrated in the step S2 to train a YOLOv4 lightweight network model, and training after 5 ten thousand iterations to obtain a YOLOv4 lightweight digital detection model (Number _ detector);
step S4, performing digital detection and identification on the multifunctional digital table picture by using the YOLOv4 lightweight digital detection model (Number _ detector) obtained in step S3, and returning information of each detected Number, specifically including the following steps:
step S40, detecting pictures shot by the inspection robot by using a YOLOv4 switch cabinet device detection model (Object _ Detector) to obtain a multifunctional digital table, and cutting the pictures from the switch cabinet pictures according to a detection rectangular frame to obtain multifunctional digital table pictures;
step S41, inputting the cut multifunctional digital chart picture into a YOLOv4 lightweight digital detection model for digital detection, and returning position rectangular frame information and category information of each number;
specifically, the position rectangular frame information is (x, y, w, h), where x denotes a left vertex abscissa, y denotes a left vertex ordinate, w denotes a width of a rectangular width, h denotes a height of a rectangular frame, and the category information is one of numbers 0,1,2,3,4,5,6,7,8, 9;
the multifunctional digital table picture is obtained by detecting a picture shot by an inspection robot through a YOLOv4 switch cabinet device detection model (Object _ Detector), the YOLOv4 switch cabinet device detection model (Object _ Detector) is a trained detection model, and the training of the YOLOv4 switch cabinet device detection model comprises the following steps:
(1) establishing a training set: manually labeling 5000 switch cabinet pictures shot by the inspection robot in multiple inspection processes by using labeling software, storing the device position and device type information in each switch cabinet picture in a corresponding XML file, and establishing a training set consisting of 5000 pictures and 5000 XML files;
(2) establishing a test set: manually labeling 1000 switch cabinet pictures which do not participate in training by using labeling software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing a test set consisting of 1000 pictures and 1000 XML files;
(3) training an original YOLOv4 target detection network model on a training set, and iteratively training for 48000 times to obtain a YOLOv4 switch cabinet device detection model (Object _ Detector);
(4) the detection performance of the YOLOv4 switch cabinet device detection model is tested on a test set.
As the inspection robot has various instrument devices in switch cabinet pictures shot at different inspection points, as shown in figure 3, besides multifunctional numbers, the switch cabinet pictures also have devices such as an indicator light, a large switch, a temperature and humidity controller and the like, the method firstly trains an original YOLOv4 target detection network model to obtain a YOLOv4 switch cabinet device detection model, then detects the multifunctional digital tables and other machines on the switch cabinet pictures by using a YOLOv4 switch cabinet device detection model (Object _ Detector) to obtain the position frame of each device and the type of the device, then cuts out the device from the switch cabinet pictures according to the position frame of each device as shown in figure 4, calls the corresponding interface function according to the type of the device to identify the working state, detects to obtain the multifunctional digital table pictures, and has high detection accuracy and high detection efficiency.
Step S5, sorting and recombining each number according to the returned information of each number to obtain a correct number arrangement order and obtain a correct reading, specifically including the following steps:
step S50, returning position rectangular frame information and category information of each number according to a YOLOv4 lightweight number detection model, sorting each number according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, and dividing the identified number into three rows, namely an upper row, a middle row and a lower row;
specifically, the YOLOv4 lightweight digital inspection model returns the position rectangular frame information and category information of each number after inspecting the multifunctional digital table, which can be expressed as { (x1, y1, w1, h1, ble1, score1), (x2, y2, w2, h2, table 2, score2), … …, (x12, y12, w12, h12, table 12, score12) }, wherein (x1, y1, w1, h1, table 1, score1) represents the position rectangular frame and category information of the first detected number, x1 and y1 represent the abscissa and ordinate of the vertex at the top left corner of the position rectangular frame, w1 and h1 represent the width and height of the position rectangular frame, table 1 represents the category of the number, score1 represents the probability value that the number belongs to a certain category, (x2, y2, w2, h2, table 2, score2) represents the rectangular frame and category information of the second detected number, and so on, there are 12 detected numbers in total;
to be more specific, instead of the correct number display order of the multifunctional digital table (the readings are divided into three rows, i.e., upper, middle and lower rows, and the 4 numbers in each row are arranged from left to right), the detection order of the YOLOv4 lightweight digital detection model for the 12 numbers on the picture of the multifunctional digital table is random, as shown in fig. 5, fig. 5 is a schematic diagram of the original detection result obtained by the YOLOv4 lightweight digital detection model, the rectangular box represents a rectangular box of the position detected by the YOLOv4 lightweight digital detection model for each number in the digital table, the number of the left vertex of the rectangular box represents the precedence order of the YOLOv4 lightweight digital detection model when detecting the 12 numbers, for example, the number 0 of the left vertex of the rectangular box represents that the 3 rd number 3 of the last row of readings is the first detected;
more specifically, each number is sorted according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, and the identified number is divided into three rows, namely, an upper row, a middle row and a lower row, specifically, the detection result (x, y, w, h, cable, score) of each number is sorted according to the size of the vertical coordinate y of the top left vertex of the 12 rectangular frames. The y value is changed from small to large, the 12-digit detection results are also rearranged according to the sequence of the y value from small to large, and 12 ascending-order digit detection results are obtained:
new_result={R1,R2,R3,R4,R5,R6,R7,R8,R9,R10,R11,R12}；
wherein R1= (x1, y1, w1, h1, cable 1, score1), because the difference of the y values of the 4 digital position frames belonging to the same row is very small, the new _ result is directly divided into 3 parts to obtain the upper, middle and lower rows corresponding to the reading in the digital table, the upper row contains four digital detection results of R1, R2, R3 and R4, the middle row contains R5, R6, R7 and R8, and the lower row contains R9, R10, R11 and R12.
And step S51, sequencing the digital detection results belonging to the same row from left to right according to the size of the abscissa of the left upper vertex of the rectangular frame at the position of the digital detection results, finishing sequencing recombination, obtaining the correct digital arrangement sequence, and obtaining the correct reading.
More specifically, to sort the 4 digital detection results belonging to the same row from left to right according to the size of the x value of the abscissa at the top left vertex of the rectangular frame, for example, for the 4 digital detection results first _ line = (R1, R2, R3, R4) belonging to the first row, the x values in R1, R2, R3, and R4 are sorted from small to large, and then R1, R2, R3, and R4 are also position-adjusted according to the order of their x values, thereby completing the left-to-right sorting of the 4 digital detection results.
After steps S50 and S51, the digit detection result is sorted and reorganized, as shown in fig. 6, the digit table reading is successfully divided into three rows, each row is composed of 4 digits, the position of each digit is represented by the serial number 0,1,2,3 (the digit at the upper left corner of the rectangular box), as shown in fig. 7, the digit at the upper left corner of the rectangular box represents the recognition result of the digit in the rectangular box.
And step S6, adding decimal points to the reading according to the preset decimal number to obtain the final reading.
Further, the readings obtained after the reordering in step S5 are only integer readings containing 4 digits, such as 1430, 1431 and 1430, but the actual readings of the multifunctional meter are 14.30, 14.31 and 14.30, and since the camera of the robot is far away from the cabinet, and the motion characteristic of the robot and the display characteristic of the multifunctional meter, it is difficult to determine the position of the decimal point of each line of the readings of the multifunctional meter through the image processing algorithm, which results in low accuracy and stability, and results in a great difference of ten times or thousand times between the readings. Therefore, the decimal digit of each multifunctional digit on the multi-switch cabinet is manually determined by the method, and the method specifically comprises the following steps:
step S60, before the inspection robot formally inspects the switch cabinet of the power distribution room, the staff performs stepping point planning of the robot motion path through a Qt upper computer interface for controlling the robot motion, namely, the inspection robot is controlled to perform sequential photographing identification on the switch cabinet;
step S61, when the inspection robot steps on the point, if the shooting point has a multifunctional number table, i.e. the decimal point number is input in the decimal point adding column of the Qt interface, for example, there are 3 multifunctional number tables in the picture shot by the shooting point, and the reading of each table contains 2 decimal points, "2, 2, 2" is input, and if the reading of the first two tables of the 3 multifunctional tables shot by the shooting point contains 3 decimal points and the reading of the last table contains 2 decimal points, "3, 3, 2" is input.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims (3)
1. A multifunctional digital meter automatic reading method based on a target detection algorithm is characterized by comprising the following steps:
step S1, carrying out lightweight improvement on the YOLOv4 network model to obtain a YOLOv4 lightweight network model, and specifically comprising the following steps:
step S10, replacing a CSPDarknet53 network structure in the original YOLOv4 network model with a lightweight network structure MobileNet V3 to serve as a backbone network;
step S11, deleting the two subsequent YOLOv3head structures in the original YOLOv4 network model, only reserving one YOLOv3head structure, and reducing the number of anchor points;
step S12, reducing the net input from 416 by 416 to 192 by 256;
step S2, collecting the picture of the multifunctional digital meter and calibrating the position and the category of the digit on the picture of the multifunctional digital meter;
in step S2, the method specifically includes the following steps:
s20, screening out pictures containing the multifunctional digital meter from a large number of switch cabinet pictures shot by the inspection robot in the inspection process, and manually framing out the position of the multifunctional digital meter area;
step S21, cutting out a single multifunctional digital table picture from the picture of the switch cabinet according to the position rectangular frame selected by the manual frame and storing the single multifunctional digital table picture;
step S22, the position calibration and the category calibration are carried out on the stored numbers on each multifunctional digital table picture by using the labeling software, and the labeling information is stored in an XML file;
step S3, training a YOLOv4 lightweight network model by using the multifunctional digital chart picture calibrated in the step S2 and the calibration information thereof to obtain a YOLOv4 lightweight digital detection model;
step S4, performing digital detection and identification on the multifunctional digital chart picture by using the YOLOv4 lightweight digital detection model obtained in step S3, and returning information of each detected number, specifically comprising the following steps:
step S40, detecting pictures shot by the inspection robot by using a YOLOv4 switch cabinet device detection model to obtain a multifunctional digital table, and cutting the pictures from the switch cabinet pictures according to a detection rectangular frame to obtain multifunctional digital table pictures; the multifunctional digital meter picture is obtained by detecting a picture shot by the inspection robot through a YOLOv4 switch cabinet device detection model;
step S41, inputting the cut multifunctional digital chart picture into a YOLOv4 lightweight digital detection model for digital detection, and returning position rectangular frame information and category information of each number;
the Yolov4 switch cabinet device detection model is a trained detection model, and the training of the Yolov4 switch cabinet device detection model comprises the following steps:
(1) establishing a training set: manually marking a plurality of switch cabinet pictures shot by the inspection robot in a multi-time inspection mode by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing a training set;
(2) establishing a test set: manually marking a plurality of switch cabinet pictures which do not participate in training by using marking software, storing the device position and the device type information in each switch cabinet picture in a corresponding XML file, and establishing to obtain a test set;
(3) training an original YOLOv4 target detection network model on a training set, and obtaining a YOLOv4 switch cabinet device detection model after iterative training;
(4) testing the detection performance of a YOLOv4 switch cabinet device detection model on a test set;
step S5, sorting and recombining each number according to the returned information of each number to obtain a correct number arrangement order and obtain a correct reading, specifically including the following steps:
step S50, returning position rectangular frame information and category information of each number according to a YOLOv4 lightweight number detection model, sorting each number according to the size of the vertical coordinate of the top left vertex of the position rectangular frame, and dividing the identified number into three rows, namely an upper row, a middle row and a lower row;
step S51, sorting the digital detection results belonging to the same row from left to right according to the size of the abscissa of the left upper vertex of the rectangular frame at the position of the digital detection results, finishing sorting and recombining, obtaining a correct digital arrangement sequence, and obtaining a correct reading;
and step S6, adding decimal points to the reading according to the preset decimal number to obtain the final reading.
2. The method for automatically reading a multifunctional digital meter based on an object detection algorithm as claimed in claim 1, wherein the step S3 is specifically as follows: and (4) training the YOLOv4 lightweight network model by using the multifunctional digital table pictures calibrated in the step S2 and the corresponding XML files, and performing iterative training to obtain a YOLOv4 lightweight digital detection model.
3. The method for automatically reading a multifunctional digital meter based on an object detection algorithm as claimed in claim 1, wherein the step S6 specifically comprises the following steps:
step S60, before the inspection robot formally inspects the switch cabinet of the power distribution room, the staff plan the stepping points of the robot motion path through the Qt upper computer interface controlling the robot motion;
and step S61, when the inspection robot moves to step on the point, if the shooting point has a multifunctional digital table, inputting the decimal point digit in the decimal point adding column of the Qt interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011585525.8A CN112308054B (en) | 2020-12-29 | 2020-12-29 | Automatic reading method of multifunctional digital meter based on target detection algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011585525.8A CN112308054B (en) | 2020-12-29 | 2020-12-29 | Automatic reading method of multifunctional digital meter based on target detection algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112308054A CN112308054A (en) | 2021-02-02 |
| CN112308054B true CN112308054B (en) | 2021-07-20 |
Family
ID=74487611
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011585525.8A Active CN112308054B (en) | 2020-12-29 | 2020-12-29 | Automatic reading method of multifunctional digital meter based on target detection algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112308054B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560816A (en) * | 2021-02-20 | 2021-03-26 | 北京蒙帕信创科技有限公司 | Equipment indicator lamp identification method and system based on YOLOv4 |
| CN113095127B (en) * | 2021-03-01 | 2022-06-17 | 兰州大学 | Building post-earthquake positioning and damage state evaluation method based on satellite images |
| CN113065556B (en) * | 2021-04-16 | 2024-04-02 | 西安交通大学 | Digital instrument positioning and digital identifying method and device and computer equipment |
| CN112990131B (en) * | 2021-04-27 | 2021-10-26 | 广东科凯达智能机器人有限公司 | Method, device, equipment and medium for acquiring working gear of voltage change-over switch |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
| CN110837825A (en) * | 2019-10-17 | 2020-02-25 | 天津大学 | Meter identification and reading system based on embedded type |
| CN111368906A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Pointer type oil level indicator reading identification method based on deep learning |
| CN211856402U (en) * | 2020-03-13 | 2020-11-03 | 苏州鑫睿益荣信息技术有限公司 | Automobile instrument panel detection system based on machine vision |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6250526B2 (en) * | 2014-11-21 | 2017-12-20 | 三菱電機ビルテクノサービス株式会社 | Weighing meter reader and program |
| WO2019223582A1 (en) * | 2018-05-24 | 2019-11-28 | Beijing Didi Infinity Technology And Development Co., Ltd. | Target detection method and system |
| CN109344850A (en) * | 2018-07-27 | 2019-02-15 | 华南理工大学 | A kind of water meter automatic testing method based on YOLO |
| CN110188662A (en) * | 2019-05-28 | 2019-08-30 | 唐山海森电子股份有限公司 | A kind of AI intelligent identification Method of water meter number |
| CN110659636B (en) * | 2019-09-20 | 2022-05-13 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
| CN111062282B (en) * | 2019-12-05 | 2023-09-29 | 武汉科技大学 | Substation pointer instrument identification method based on improved YOLOV3 model |
| CN111353502B (en) * | 2020-02-25 | 2023-06-30 | 北京眸视科技有限公司 | Digital table identification method and device and electronic equipment |
| CN112052826A (en) * | 2020-09-18 | 2020-12-08 | 广州瀚信通信科技股份有限公司 | Intelligent enforcement multi-scale target detection method, device and system based on YOLOv4 algorithm and storage medium |
-
2020
- 2020-12-29 CN CN202011585525.8A patent/CN112308054B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | A kind of image detecting method, device and computer readable storage medium |
| CN110837825A (en) * | 2019-10-17 | 2020-02-25 | 天津大学 | Meter identification and reading system based on embedded type |
| CN111368906A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Pointer type oil level indicator reading identification method based on deep learning |
| CN211856402U (en) * | 2020-03-13 | 2020-11-03 | 苏州鑫睿益荣信息技术有限公司 | Automobile instrument panel detection system based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112308054A (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112308054B (en) | Automatic reading method of multifunctional digital meter based on target detection algorithm | |
| CN108686978B (en) | ARM-based fruit category and color sorting method and system | |
| CN103076589B (en) | A kind of digital multimeter automatic calibrator and calibration method thereof | |
| CN108648169A (en) | The method and device of high voltage power transmission tower defects of insulator automatic identification | |
| CN110991435A (en) | Express waybill key information positioning method and device based on deep learning | |
| CN106022345B (en) | A kind of high voltage isolator state identification method based on Hough forest | |
| CN109451532A (en) | A kind of check method and device of base station location | |
| CN107229560A (en) | A kind of interface display effect testing method, image specimen page acquisition methods and device | |
| CN109711377A (en) | Standardize examinee's positioning and method of counting in the single-frame images of examination hall monitoring | |
| CN111738229B (en) | Automatic reading method for scale of pointer dial | |
| CN116256908B (en) | Calibration method based on miniLED backlight module | |
| CN110647941A (en) | Vegetable identification method and equipment based on convolutional neural network | |
| CN105719236B (en) | The method for generating complete high-resolution vegetation cover degree image seasonal variations sequence | |
| CN108549840A (en) | A kind of method and device that analog instrument simultaneously and rapidly identifies | |
| CN107945079A (en) | A kind of poverty alleviation object selection method and device | |
| CN113076730A (en) | Test report intelligent management method, device, system and storage medium | |
| CN101593479B (en) | Method for testing avionic display | |
| CN112733845A (en) | Interest area problem identification method, interest area inspection method and device | |
| CN104574455B (en) | Image rebuilding method and device | |
| CN106696998A (en) | Method, device and equipment for monitoring deviation of rails | |
| CN114037993B (en) | Substation pointer instrument reading method and device, storage medium and electronic equipment | |
| CN105955918A (en) | Experimental multi-instrument system | |
| CN115922697A (en) | Intelligent robot automatic inspection method based on transformer substation digital twinning technology | |
| CN110909653B (en) | Method for automatically calibrating screen cabinet of distribution room by indoor robot | |
| CN203149105U (en) | Digital universal meter automatic calibrating device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Automatic reading method of multi-function digital meter based on target detection algorithm Effective date of registration: 20220527 Granted publication date: 20210720 Pledgee: Agricultural Bank of China Limited Shunde Daliang sub branch Pledgor: GUANGDONG KEYSTAR INTELLIGENCE ROBOT Co.,Ltd. Registration number: Y2022980006520 |