CN112348018A - Digital display type instrument reading identification method based on inspection robot - Google Patents

Digital display type instrument reading identification method based on inspection robot Download PDF

Info

Publication number
CN112348018A
CN112348018A CN202011280084.0A CN202011280084A CN112348018A CN 112348018 A CN112348018 A CN 112348018A CN 202011280084 A CN202011280084 A CN 202011280084A CN 112348018 A CN112348018 A CN 112348018A
Authority
CN
China
Prior art keywords
digital display
picture
display area
instrument
digital
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011280084.0A
Other languages
Chinese (zh)
Inventor
王士兴
赵妍
黄玉龙
邱垚
张兆康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ansen Intelligent Information Technology Co ltd
Original Assignee
Hangzhou Ansen Intelligent Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Ansen Intelligent Information Technology Co ltd filed Critical Hangzhou Ansen Intelligent Information Technology Co ltd
Priority to CN202011280084.0A priority Critical patent/CN112348018A/en
Publication of CN112348018A publication Critical patent/CN112348018A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/02Recognising information on displays, dials, clocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition

Abstract

Firstly, acquiring a digital display instrument picture through a visible light camera carried by a cloud deck of the inspection robot and an image video management process; then, an external rectangle of the instrument dial is intercepted as a template image; thirdly, intercepting the digital display area and obtaining position information, segmenting the number in the digital display area, and recording the decimal point digit; fourthly, acquiring a picture to be detected and finishing initialization; fifthly, template matching and correction are carried out on the instrument position of the picture to be detected; sixthly, intercepting the digital display area picture to perform gray processing, judging the difference between the foreground font and the background, performing self-adaptive binarization on the digital display area, performing single digital segmentation on the digital display area, and refining the outline of the digital display area; and converting the single number into a pixel, storing the pixel, generating a data format required by the classifier, predicting and outputting a result. The invention improves the identification speed and the universality of the intelligent inspection robot on the digital display type instrument reading in an applicable scene.

Description

Digital display type instrument reading identification method based on inspection robot
Technical Field
The invention belongs to the technical field of image processing and recognition, and relates to a digital display type instrument reading recognition method based on an inspection robot.
Background
With the development and breakthrough of the robot technology, the intelligent inspection robot can be used for monitoring the field operation condition in the industrial fields of oil fields, transformer substations, natural gas stations and the like, wherein the digital display instrument is generally used for various field measurements and monitoring. The traditional mode of patrolling and examining adopts the manual work to carry out reading and manual record to the instrument, nevertheless compares in artifical reading because the particularity and the extremely end environment of being suitable for the scene, and the intelligence patrols and examines the robot and is more convenient and accurate to the reading of digital display instrument. The image of the digital display instrument is obtained by controlling the angle of the holder on the robot, and the image is processed and subjected to target detection to obtain the reading of the instrument, so that the identification process is more intelligent.
On the one hand, the cost of manual inspection reading and recording is higher, the environment influence that locates leads to patrolling and examining personnel's safety can't obtain guaranteeing, and human eyes also have certain error simultaneously, lead to recognition efficiency and rate of accuracy lower. On the other hand, due to the fact that industrial field scenes are complex, the shapes, backgrounds and the like of meters on each field are different greatly, the existing identification method is too slow in identification speed and poor in universality, and therefore the fact that the intelligent inspection robot based on the digital display type intelligent identification method has very important significance.
Disclosure of Invention
The invention aims to provide a digital display type meter reading identification method based on an inspection robot, so as to improve the identification speed and the universality of the intelligent inspection robot on the digital display type meter reading in an applicable scene.
The invention adopts the following technical scheme:
a digital display type instrument reading identification method based on an inspection robot comprises the following steps:
step 1: acquiring a digital display instrument picture through a visible light camera carried by an intelligent inspection robot holder and an image video management process deployed by a robot body;
step 2: on the picture obtained in the step 1, manually intercepting the minimum external rectangle of the instrument dial, saving the position information of the external rectangle on the whole picture as a template image, and generating a corresponding template file I;
and step 3: manually intercepting a digital display area of the instrument dial to obtain position information of the digital display area on the picture obtained in the step 1, dividing the number in the digital display area, storing the position information of a single number on the picture obtained in the step 1, recording the decimal point number of the single number, and generating a template file II;
and 4, step 4: the inspection robot executes tasks and obtains a picture to be detected, smooth filtering pretreatment is carried out on the picture, the size of the picture is modified into a size which needs to be unified for subsequent identification, and initialization is completed;
and 5: according to the dial plate characteristics in the template file I, template matching is carried out on the instrument position of the picture to be detected, the matched optimal position is returned, and dial plate position information on the picture to be detected is positioned; mapping the position information of the digital display area preset in the template file II to the picture to be detected, and then correcting the digital display area;
step 6: performing gray level processing (reducing the original data amount of an image and facilitating subsequent processing) on a digital display area image intercepted from a to-be-detected image, judging the difference between a foreground font and a background according to a threshold preset in modeling, selecting a binary threshold or an inverse binary threshold, and performing self-adaptive binarization on a digital display area; using a template file II which reserves a single digital position before as prior information, carrying out single digital segmentation on the logarithmic display area and thinning the outline of the logarithmic display area;
and 7: converting a single digital picture after being segmented into pixels for storage, generating a data format required by a classifier, putting the pixel into a libsvm classifier trained by a large number of samples in advance for prediction, and outputting a digital result which is in accordance with the order of the number of the instrument bits;
and 8: adding decimal points to the digital results classified by using the classifier in combination with the preset decimal point number during modeling, and performing allowance processing on the digital adhesion condition which possibly occurs;
and step 9: and returning the identification result, and finishing the identification task.
Further, the process of acquiring the digital display instrument picture in the step 1 is as follows: the intelligent inspection robot adjusts the angle of the holder, the adjusting process refers to a distributed processing method for robot cluster image identification, and then an image acquisition module is called to acquire an image so as to acquire a digital display instrument picture.
Further, the image acquisition module in step 1 acquires an RGB image.
Further, the template matching process performed on the instrument position in step 5 is as follows: and extracting the pixel characteristics of the dial from the template image, matching the pixel characteristics with the dial in the picture to be detected, and calculating the correlation coefficient of the template image and the picture to be detected, wherein the rectangular frame corresponding to the maximum coefficient returns the best matching position.
Further, in step 6, the process of selecting the binary threshold or the inverse binary threshold is as follows: according to a threshold T set in the previous modeling, if the threshold T is greater than 0, selecting a binary threshold type, namely setting the gray value of a pixel point greater than the threshold to be 255, and otherwise, setting the gray value to be 0; if the threshold T is less than 0, selecting an inverse binary threshold, namely setting the gray value of the pixel point greater than the threshold as 0, otherwise, setting the gray value as 255.
Further, in step 6, the process of performing single digital segmentation on the logarithmic display region and refining the contour thereof is as follows: correcting the digital display area by affine transformation according to the position of the digital display area stored by modeling and the position information manually intercepted by a single digit, and segmenting the single digit; and performing morphological operation on the separated single number.
The invention has the beneficial effects that:
the identification method of the invention adopts a pre-modeling mode, and only carries out template matching on the dial plate in the picture compared with the prior identification method which adopts the whole picture to carry out template matching, thereby greatly reducing the problem of overlong identification time caused by template matching in time.
In the identification process, a libsvm classifier is used for identifying and predicting the numbers, the traditional KNN is usually adopted for classifying the numbers in the identification field, the libsvm classifier has the characteristics of small program, less input parameters and flexibility, classification results can be directly output according to the digit order, and the calculation complexity is reduced.
Because the digital display instrument is usually used in industrial stations with complex environments, the environments of the stations are greatly different, the outlines of fonts and the like of the digital display instrument are also different, the digital display instrument can be identified no matter how the scene changes by adopting methods of modeling in advance and local matching of dial plates, and the libsvm classifier is not limited by the font color, so that the universality of identification of the digital display instrument of each station and each shape is greatly improved. In conclusion, the method can obviously improve the identification precision and stability of the digital display instrument.
Drawings
FIG. 1 is a logic flow diagram of an identification method of the present invention;
FIG. 2 is a picture of a template image K in an embodiment;
FIG. 3 is a preprocessed image to be detected;
FIG. 4 is a schematic diagram of a matching process;
FIG. 5 is a digital display area captured in a detected picture;
fig. 6 is a single number of divisions.
Detailed Description
The invention is described in further detail below with reference to the figures and the detailed description.
The invention discloses an intelligent inspection robot-based digital display type instrument intelligent identification method, which is used for processing a digital display type instrument to be identified as shown in figure 1, and is implemented according to the following steps:
step 1: acquiring a digital display instrument picture through a visible light camera carried by an intelligent inspection robot holder and an image video management process deployed by a robot body; the process of acquiring the digital display instrument picture comprises the following steps: the intelligent inspection robot adjusts the angle of the pan/tilt head (see patent of distributed processing method for robot cluster image recognition), and then calls an image acquisition module to acquire an image, wherein the module can acquire an RGB image, adjust the position area image of the target instrument to the center of the field of view of the camera, and acquire a digital display instrument picture S1, wherein the visible light camera selects a seaman camera 3007c to be 1/2.8 inches.
Step 2: manually capturing the minimum circumscribed rectangle of the instrument dial from the obtained S1 picture, storing the position information of the rectangle, wherein xyz of the rectangle is (259,198,142,140), and the rectangle is used as a template image K, as shown in FIG. 2, and generating a corresponding template file 1;
and step 3: manually intercepting the digital display area to obtain coordinate information of 4 vertexes of the digital display area in S1, wherein the coordinate information is respectively (367,234), (514,234), (367, 342) and (514,342), the coordinate information is sequentially divided into single numbers in the digital display area, the minimum circumscribed rectangle position information of each number is stored, and the minimum circumscribed rectangle position information is respectively (8,4,13,31), (23,5,12,30) (34,3,14,32) (48,2,12,33), and a template file 2 is generated;
and 4, step 4: when the identification instruction is received, the intelligent inspection robot acquires a picture to be detected S2, preprocesses the picture, as shown in FIG. 3, and presets the size for the picture.
The preprocessing part of the picture comprises: the RGB image is converted into a gray scale space, the characteristics of a target object are highlighted, filtering and smoothing processing are carried out, and a noise part in the image is filtered. The picture preset size is (416 ).
And 5: matching the position of the instrument, and according to the characteristic information in the template image K and the inspection picture S2 to be detected, calculating the correlation coefficient of S2 and K, wherein the rectangular frame corresponding to the maximum value of the coefficient returns the best position for matching; and mapping the preset position information of the digital display area to the picture to be detected S2, and correcting the digital display area by affine transformation as shown in FIG. 4.
Step 6: the digital display area intercepted from the detection picture is subjected to gray processing, the difference between the foreground font and the background (the threshold value is black and white when the threshold value is less than 0, and is white and black when the threshold value is greater than 0) is judged according to a threshold value-155 preset in the modeling, a binary threshold value or an inverse binary threshold value is selected, the inverse binary threshold value is selected when the threshold value is less than 0, and the self-adaptive binarization of the digital display area is completed, as shown in fig. 5. A template file II which reserves a single digital position before is used as prior information; according to the position of the digital display area stored by modeling and the position information manually intercepted by a single digit, correcting the digital display area by affine transformation, and segmenting the single digit as shown in FIG. 6; obtaining the outline of the number by recycling morphological operation on the single number which is divided;
and 7: and converting the single digital picture after the segmentation processing into pixels for storage, generating a data format required by a classifier, putting the pixels into a libsvm classifier trained by a large number of samples in advance for prediction, and outputting a digital result (9,6,1,5) in accordance with the order of the meter digits.
And 8: combining the digital results classified by using the classifier with the decimal point number preset during modeling, wherein the decimal point number is 2, adding decimal points, if the output digital results are increased due to adhesion, adopting single digit surplus processing, and directly combining the decimal point number to directly give a recognition result of 96.15, wherein the output result does not have adhesion;
and step 9: and returning the identification result, and finishing the identification task.
The invention adopts the preset rectangular area, the digital display area and the position of a single digit of the meter dial as prior information, then carries out template matching, feature extraction and positioning registration on the digital display meter picture to be identified, and finally predicts the digit by using a classifier. The method can solve the problem that the calculation time is too long in the process of positioning the dial plate position in the prior art, accurately obtains the reading of the digital display instrument, has high stability and strong noise interference resistance, and can obtain accurate and stable digital reading by identifying the digital display instrument by using the method provided by the invention.

Claims (6)

1. A digital display type instrument reading identification method based on an inspection robot is characterized by comprising the following steps:
step 1: acquiring a digital display instrument picture through a visible light camera carried by an intelligent inspection robot holder and an image video management process deployed by a robot body;
step 2: on the picture obtained in the step 1, manually intercepting the minimum external rectangle of the instrument dial, saving the position information of the external rectangle on the whole picture as a template image, and generating a corresponding template file I;
and step 3: manually intercepting a digital display area of the instrument dial to obtain position information of the digital display area on the picture obtained in the step 1, dividing the number in the digital display area, storing the position information of a single number on the picture obtained in the step 1, recording the decimal point number of the single number, and generating a template file II;
and 4, step 4: the inspection robot executes tasks and obtains a picture to be detected, smooth filtering pretreatment is carried out on the picture, the size of the picture is modified into a size which needs to be unified for subsequent identification, and initialization is completed;
and 5: according to the dial plate characteristics in the template file I, template matching is carried out on the instrument position of the picture to be detected, the matched optimal position is returned, and dial plate position information on the picture to be detected is positioned; mapping the position information of the digital display area preset in the template file II to the picture to be detected, and then correcting the digital display area;
step 6: performing gray level processing (reducing the original data amount of an image and facilitating subsequent processing) on a digital display area image intercepted from a to-be-detected image, judging the difference between a foreground font and a background according to a threshold preset in modeling, selecting a binary threshold or an inverse binary threshold, and performing self-adaptive binarization on a digital display area; using a template file II which reserves a single digital position before as prior information, carrying out single digital segmentation on the logarithmic display area and thinning the outline of the logarithmic display area;
and 7: converting a single digital picture after being segmented into pixels for storage, generating a data format required by a classifier, putting the pixel into a libsvm classifier trained by a large number of samples in advance for prediction, and outputting a digital result which is in accordance with the order of the number of the instrument bits;
and 8: adding decimal points to the digital results classified by using the classifier in combination with the preset decimal point number during modeling, and performing allowance processing on the digital adhesion condition which possibly occurs;
and step 9: and returning the identification result, and finishing the identification task.
2. The inspection robot-based digital display instrument reading identification method according to claim 1, wherein the process of acquiring the digital display instrument picture in the step 1 is as follows: the intelligent inspection robot adjusts the angle of the holder, the adjusting process refers to a distributed processing method for robot cluster image identification, and then an image acquisition module is called to acquire an image so as to acquire a digital display instrument picture.
3. The inspection robot-based digital display meter reading identification method according to claim 1, wherein the image acquisition module of step 1 acquires RGB images.
4. The inspection robot-based digital display meter reading identification method according to claim 1, wherein the template matching process performed on the meter position in the step 5 is as follows: and extracting the pixel characteristics of the dial from the template image, matching the pixel characteristics with the dial in the picture to be detected, and calculating the correlation coefficient of the template image and the picture to be detected, wherein the rectangular frame corresponding to the maximum coefficient returns the best matching position.
5. The inspection robot-based digital display meter reading identification method according to claim 1, wherein in step 6, the process of selecting the binary threshold or the inverse binary threshold is as follows: according to a threshold T set in the previous modeling, if the threshold T is greater than 0, selecting a binary threshold type, namely setting the gray value of a pixel point greater than the threshold to be 255, and otherwise, setting the gray value to be 0; if the threshold T is less than 0, selecting an inverse binary threshold, namely setting the gray value of the pixel point greater than the threshold as 0, otherwise, setting the gray value as 255.
6. The inspection robot-based digital display instrument reading identification method according to claim 1, wherein in the step 6, the process of performing single digital segmentation on the digital display area and refining the outline of the digital display area comprises the following steps: correcting the digital display area by affine transformation according to the position of the digital display area stored by modeling and the position information manually intercepted by a single digit, and segmenting the single digit; and performing morphological operation on the separated single number.
CN202011280084.0A 2020-11-16 2020-11-16 Digital display type instrument reading identification method based on inspection robot Pending CN112348018A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011280084.0A CN112348018A (en) 2020-11-16 2020-11-16 Digital display type instrument reading identification method based on inspection robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011280084.0A CN112348018A (en) 2020-11-16 2020-11-16 Digital display type instrument reading identification method based on inspection robot

Publications (1)

Publication Number Publication Date
CN112348018A true CN112348018A (en) 2021-02-09

Family

ID=74363891

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011280084.0A Pending CN112348018A (en) 2020-11-16 2020-11-16 Digital display type instrument reading identification method based on inspection robot

Country Status (1)

Country Link
CN (1) CN112348018A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113159027A (en) * 2021-04-13 2021-07-23 杭州电子科技大学 Seven-segment type digital display instrument identification method based on minimum external rectangle variant
CN113778091A (en) * 2021-09-13 2021-12-10 华能息烽风力发电有限公司 Method for inspecting equipment of wind power plant booster station
CN116189166A (en) * 2023-02-07 2023-05-30 台州勃美科技有限公司 Meter reading method and device and robot

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529537A (en) * 2016-11-22 2017-03-22 亿嘉和科技股份有限公司 Digital meter reading image recognition method
CN108573261A (en) * 2018-04-17 2018-09-25 国家电网公司 A kind of read out instrument recognition methods suitable for Intelligent Mobile Robot
CN109034160A (en) * 2018-07-06 2018-12-18 江苏迪伦智能科技有限公司 A kind of mixed decimal point digital instrument automatic identifying method based on convolutional neural networks
CN110321890A (en) * 2019-04-26 2019-10-11 重庆邮电大学 A kind of digital instrument recognition methods of electric inspection process robot
CN111539330A (en) * 2020-04-17 2020-08-14 西安英诺视通信息技术有限公司 Transformer substation digital display instrument identification method based on double-SVM multi-classifier

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529537A (en) * 2016-11-22 2017-03-22 亿嘉和科技股份有限公司 Digital meter reading image recognition method
CN108573261A (en) * 2018-04-17 2018-09-25 国家电网公司 A kind of read out instrument recognition methods suitable for Intelligent Mobile Robot
CN109034160A (en) * 2018-07-06 2018-12-18 江苏迪伦智能科技有限公司 A kind of mixed decimal point digital instrument automatic identifying method based on convolutional neural networks
CN110321890A (en) * 2019-04-26 2019-10-11 重庆邮电大学 A kind of digital instrument recognition methods of electric inspection process robot
CN111539330A (en) * 2020-04-17 2020-08-14 西安英诺视通信息技术有限公司 Transformer substation digital display instrument identification method based on double-SVM multi-classifier

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113159027A (en) * 2021-04-13 2021-07-23 杭州电子科技大学 Seven-segment type digital display instrument identification method based on minimum external rectangle variant
CN113159027B (en) * 2021-04-13 2024-02-09 杭州电子科技大学 Seven-segment digital display instrument identification method based on minimum external rectangular variant
CN113778091A (en) * 2021-09-13 2021-12-10 华能息烽风力发电有限公司 Method for inspecting equipment of wind power plant booster station
CN116189166A (en) * 2023-02-07 2023-05-30 台州勃美科技有限公司 Meter reading method and device and robot

Similar Documents

Publication Publication Date Title
CN112348018A (en) Digital display type instrument reading identification method based on inspection robot
CN106096610B (en) A kind of file and picture binary coding method based on support vector machines
CN108133216B (en) Nixie tube reading identification method capable of realizing decimal point reading based on machine vision
CN112149543B (en) Building dust recognition system and method based on computer vision
CN110309806B (en) Gesture recognition system and method based on video image processing
CN111797766B (en) Identification method, identification device, computer-readable storage medium, and vehicle
CN111539330B (en) Transformer substation digital display instrument identification method based on double-SVM multi-classifier
CN112561899A (en) Electric power inspection image identification method
CN112907519A (en) Metal curved surface defect analysis system and method based on deep learning
CN113324864A (en) Pantograph carbon slide plate abrasion detection method based on deep learning target detection
CN110807775A (en) Traditional Chinese medicine tongue image segmentation device and method based on artificial intelligence and storage medium
CN113034474A (en) Test method for wafer map of OLED display
CN112861654A (en) Famous tea picking point position information acquisition method based on machine vision
CN114241438B (en) Traffic signal lamp rapid and accurate identification method based on priori information
CN111783773A (en) Correction method for angle-oriented inclined wire pole signboard
CN112184816A (en) Lithium battery flexible positioning method based on battery tabs
CN115082776A (en) Electric energy meter automatic detection system and method based on image recognition
CN113642570A (en) Method for recognizing license plate of mine car in dark environment
CN113205099A (en) Visual detection system and method for material-removing characters on surface of nickel-plated metal
CN110930358B (en) Solar panel image processing method based on self-adaptive algorithm
CN111738264A (en) Intelligent acquisition method for data of display panel of machine room equipment
CN114820707A (en) Calculation method for camera target automatic tracking
CN113689365B (en) Target tracking and positioning method based on Azure Kinect
CN112532938B (en) Video monitoring system based on big data technology
CN115457459A (en) Machine vision system capable of effectively improving detection efficiency

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210209