CN110659636A - Pointer instrument reading identification method based on deep learning - Google Patents
Pointer instrument reading identification method based on deep learning Download PDFInfo
- Publication number
- CN110659636A CN110659636A CN201910894088.9A CN201910894088A CN110659636A CN 110659636 A CN110659636 A CN 110659636A CN 201910894088 A CN201910894088 A CN 201910894088A CN 110659636 A CN110659636 A CN 110659636A
- Authority
- CN
- China
- Prior art keywords
- pointer
- scale mark
- instrument
- straight line
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/1475—Inclination or skew detection or correction of characters or of image to be recognised
- G06V30/1478—Inclination or skew detection or correction of characters or of image to be recognised of characters or characters lines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a pointer instrument reading identification method based on deep learning, which identifies a dial plate area in an instrument picture to be identified through a first deep network model; the area where the dial plate of the instrument to be identified is located is cut out from the picture; recognizing the coordinates of the circle center, the digital coordinates of each scale mark, the coordinates of the needle point of the half pointer and the coordinates of the two scale mark numbers positioned at the two sides of the pointer in the dial through a second depth network model; respectively intercepting the areas where the two scale mark numbers are located from the intercepted picture of the dial plate area of the instrument to be identified; and identifying the numerical values of the two scale mark numbers through a third depth network model, and reading by adopting an angle method. The pointer instrument reading identification method based on deep learning can identify the readings of various pointer instruments at the same time, and the reading efficiency is higher.
Description
Technical Field
The invention relates to the technical field of intelligent identification of instruments and meters, in particular to a pointer instrument reading identification method based on deep learning.
Background
The pointer instrument is used as a measuring instrument, has the characteristics of simple structure, low price, dust and water prevention, strong anti-electromagnetic interference capability and the like, and is widely applied to the fields of electric power systems, petrochemical industry, aerospace aviation and the like. At present, most pointer instruments are subjected to reading interpretation by human eyes, the accuracy is low, the workload is large, the efficiency is low in the human eye identification process, and the pointer instruments are not suitable for human eyes to identify in high-temperature, high-pressure and high-radiation environments such as a transformer substation, so that researches on reading identification of the pointer instruments by replacing manual work with machines are necessary.
The reading identification of the pointer instrument mainly comprises three steps: meter positioning, dial element extraction and number interpretation. The instrument positioning mainly comprises Hough circle transformation detection and template matching methods; the dial element extraction mainly comprises Hough transformation, a subtraction method, a central projection method, a least square method, a region growing method and the like; the reading interpretation mainly comprises a distance method and an angle method. In instrument positioning, the Hough transformation circle detection and feature matching method is low in efficiency and low in speed. The central projection method, the subtraction method and the region growing method during dial element extraction have high requirements on acquiring instrument images, are easily influenced by a shooting environment, have poor robustness, and have large calculation amount and high error of Hough transformation and the least square method, so that the speed of extracting a pointer is low. In the method for reading the reading number of the instrument, the error of an angle method is large, and the calculated amount of a distance method is large. In addition, the inventor also finds that the current meter reading identification methods only perform reading identification on one type of meter and cannot identify multiple types of meters at the same time.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
Disclosure of Invention
The invention aims to provide a pointer instrument reading identification method based on deep learning, which can simultaneously identify the readings of multiple types of pointer instruments and has higher reading efficiency.
In order to achieve the above object, the present invention provides a reading identification method for a pointer instrument based on deep learning, which is used for reading identification of the pointer instrument, and comprises the following steps: inputting a picture of the instrument to be recognized into a first depth network model so as to obtain the type of the instrument to be recognized and the area coordinate of the dial plate of the instrument to be recognized in the picture; the area where the dial plate of the instrument to be identified is located is cut out from the picture; inputting the intercepted picture of the dial plate area of the instrument to be recognized into a second depth network model so as to obtain the circle center coordinate of the dial plate of the instrument to be recognized and the coordinates of each scale mark number, determining the coordinate of the half pointer point of the dial plate of the instrument to be recognized and the straight line where the pointer is located, and determining the area of two scale mark numbers closest to the straight line where the pointer is located; respectively intercepting the areas where the two scale mark numbers are located from the intercepted picture of the dial plate area of the instrument to be identified; and respectively carrying out inclination correction on the areas where the two cut scale mark numbers are located by using a contour method, then dividing the areas into digital characters by using a projection method, sequentially inputting the divided digital characters into a third depth network model so as to identify the numerical values of the two scale mark numbers, and calculating the reading of the instrument by using an angle method.
In an embodiment of the present invention, the pointer instrument reading identification method further includes constructing the first depth network model, where constructing the first depth network model includes: collecting a picture set of each instrument in a monitoring range through a visible light camera, wherein the picture set comprises pictures of each instrument shot under a plurality of shooting angles, a plurality of lighting conditions, a plurality of backgrounds and a plurality of pointer rotation angles; marking the dial plate area and the type of each instrument in the picture set, and recording the position information of the dial plate area of each instrument and the position information of the scale mark digital area of each instrument; and training the marked picture set through a target detection network model to obtain the first deep network model.
In an embodiment of the present invention, the pointer instrument reading identification method further includes constructing the second depth network model, where constructing the second depth network model includes: according to the position information of the dial area of each instrument, the dial area of each instrument is cut out from the picture of each instrument so as to obtain a dial picture set; marking a half pointer area, a dial circle center area and a scale mark number area of each dial in the dial picture set, wherein the half pointer area is a rectangular frame, and the half pointer is marked on a diagonal line of the rectangular frame; and training the marked dial plate picture set through the target detection network model so as to obtain the second deep network model.
In an embodiment of the present invention, the area for collectively labeling the half pointer area, the dial center area, and the scale mark number of each dial in the dial picture includes: and determining the half pointer, the dial center and the scale mark area of the scale mark number of each dial according to the K mean value algorithm.
In an embodiment of the present invention, the pointer instrument reading identification method further includes constructing the third depth network model, where constructing the third depth network model includes: according to the position information of the scale mark digital areas of the instruments, the scale mark digital areas of the instruments are intercepted from the pictures of the dials to obtain a scale mark digital picture set; adopting a contour method to perform inclination correction on each scale mark digit of the scale mark digit picture set, and segmenting each corrected scale mark digit by a projection method to obtain a digit character picture set; and training the digital character picture set by adopting CaffeNet network model migration learning MNIST data set weight so as to obtain the third deep network model.
In an embodiment of the present invention, the determining coordinates of the half pointer tip of the instrument dial includes: determining Euclidean distances from the center coordinates of the dial plate of the instrument to be identified to four vertexes of the semi-pointer rectangular frame; and selecting the vertex coordinate corresponding to the maximum Euclidean distance as the coordinate of the half pointer tip.
In an embodiment of the present invention, the determining the two numbers of the graduation marks closest to the straight line where the pointer is located includes: determining Euclidean distances from coordinates of each scale mark number of the instrument to be identified to coordinates of the needle point of the half pointer; and selecting the scale mark numbers corresponding to the minimum Euclidean distance value and the second minimum distance value as the two scale mark numbers closest to the straight line where the pointer is located.
In an embodiment of the present invention, the calculating the reading of the meter by using the angle method includes: respectively connecting two scale mark numbers closest to the straight line where the pointer is located with the center of a circle of the dial plate of the instrument to be identified so as to obtain a first straight line and a second straight line; determining a first included angle theta 1 between the first straight line and the straight line where the pointer is located, and determining a second included angle theta 2 between the second straight line and the straight line where the pointer is located; determining the reading v of the meter to be identified according to an angle method, wherein the angle method comprises the following steps:
wherein v1 is the value of the smaller of the two tick mark numbers closest to the line on which the pointer lies; v2 is the value of the larger of the two tick mark numbers closest to the line on which the pointer lies.
In an embodiment of the present invention, the dividing the area where the two cut-out scale line numbers are located into the number characters by a projection method after the area is respectively subjected to the slope correction by the contour method, and sequentially inputting the divided number characters into the third depth network model to identify the numerical values of the two scale line numbers closest to the straight line where the pointer is located, includes: if the numerical values of the two scale mark numbers closest to the straight line where the pointer is located cannot be identified, judging that the two scale mark numbers are shielded by the pointer of the instrument to be identified, and then confirming the areas of the two scale mark numbers closest to the straight line where the pointer is located in the rest scale mark numbers except the two scale mark numbers in the dial plate of the instrument to be identified; respectively intercepting the areas of two scale mark numbers which are closest to the straight line where the pointer is located in the rest of scale mark numbers from the intercepted picture of the dial plate area of the instrument to be identified; respectively carrying out inclination correction on the areas of the two scale mark numbers which are closest to the straight line where the pointer is located in the intercepted rest scale mark numbers by using a contour method, then segmenting the areas into digital characters by using a projection method, and sequentially inputting the segmented digital characters into a third depth network model so as to identify the numerical values of the two scale mark numbers which are closest to the straight line where the pointer is located in the rest scale mark numbers; and determining the numerical values of the two scale mark numbers closest to the straight line of the pointer according to the numerical values of the two scale mark numbers closest to the straight line of the pointer in the rest scale mark numbers.
In an embodiment of the present invention, the dividing the area where the two cut-out scale line numbers are located into the number characters by a projection method after the area is respectively subjected to the slope correction by the contour method, and sequentially inputting the divided number characters into the third depth network model to identify the numerical values of the two scale line numbers closest to the straight line where the pointer is located, includes: if one of the numerical values of the two scale mark numbers closest to the straight line where the pointer is located cannot be identified, judging that the scale mark number which cannot be identified in the two scale mark numbers is shielded by the pointer of the instrument to be identified, and then confirming the area of one scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers except the two scale mark numbers in the dial plate of the instrument to be identified; intercepting the area of one graduation mark number closest to the straight line where the pointer is located in the rest graduation mark numbers from the intercepted picture of the dial plate area of the instrument to be identified; the area of one of the cut-off scale mark numbers which is closest to the straight line where the pointer is located is subjected to slope correction by using a contour method and then is divided into digital characters by using a projection method, and the divided digital characters are sequentially input into a third depth network model, so that the numerical value of one of the cut-off scale mark numbers which is closest to the straight line where the pointer is located is identified; and determining the numerical values of the two scale mark numbers closest to the straight line where the pointer is located according to the numerical value of the scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers and the numerical value of the previously identified scale mark number.
Compared with the prior art, according to the pointer instrument reading identification method based on deep learning, the instrument type and dial plate position information of an instrument to be identified are obtained through the first depth network model, the digital position information on a half pointer, a dial plate circle center and a scale mark is obtained through the second depth network model, digital identification on the scale mark of the dial plate is realized through the third depth network model, reading of the pointer instrument is realized through training the three models, the reading efficiency is higher, and various types of pointer instruments can be identified; in addition, in the process of establishing the model, the sampled picture sample data are pictures with different shooting angles, different illumination conditions, different backgrounds and different types, so that the method can accurately read the instrument under the conditions of complex background, uneven illumination, image blurring, mirror reflection, instrument inclination and the like, and the method has good robustness and high universality; compared with the angle reading in the prior art, the angle reading method of the invention saves the step of instrument inclination correction, effectively improves the speed of instrument positioning and dial element extraction, improves the reading precision, saves the step of image preprocessing and further improves the speed of intelligent reading of the instrument compared with the method of instrument reading identification by image processing in the prior art.
Drawings
FIG. 1 is a block diagram of the steps of a depth-based pointer reading identification method according to an embodiment of the present invention;
FIG. 2 is a cut-away schematic view of a dial according to an embodiment of the invention;
FIG. 3 is a schematic diagram of the circle center, the pointer and the scale mark number according to an embodiment of the invention.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
In order to overcome the problems in the prior art, the embodiment provides a pointer type reading identification method based on depth science, reading identification is performed through three depth network models, reading efficiency is higher, the method can be suitable for reading of various pointer type instruments, robustness is high, and universality is high.
Fig. 1 is a step composition of a pointer instrument reading identification method based on deep learning according to the present embodiment. The method includes steps S1 to S5.
In step S1, a dial area in the meter picture to be recognized is identified by the first depth network model.
Specifically, a picture of the meter to be recognized is input into the first deep network model, so that the type of the meter to be recognized and the area coordinates of the dial plate of the meter to be recognized in the picture are obtained.
In step S2, the area where the dial of the meter to be recognized is located is extracted from the picture.
In step S3, the circle center coordinates, the respective scale line number coordinates, the half pointer tip coordinates, and the areas of the two scale line numbers located on both sides of the pointer in the dial are identified by the second depth network model.
Specifically, the cut-out picture of the dial area of the instrument to be recognized is input into the second depth network model, so that the circle center coordinate of the dial of the instrument to be recognized and the coordinates of each scale mark number are obtained, the coordinate of the half pointer point of the dial of the instrument to be recognized and the straight line where the pointer is located are determined, and the area of the two scale mark numbers closest to the straight line where the pointer is located is determined.
Wherein, confirm the coordinate of half pointer point of this instrument dial plate includes: determining Euclidean distances from the center coordinates of the dial plate of the instrument to be identified to four vertexes of the semi-pointer rectangular frame; and selecting the vertex coordinate corresponding to the maximum Euclidean distance as the coordinate of the half pointer tip.
The area for determining the two tick mark numbers closest to the straight line on which the pointer is located includes: determining Euclidean distances from coordinates of each scale mark number of the instrument to be identified to coordinates of the needle point of the half pointer; and selecting the area of the scale mark numbers corresponding to the Euclidean distance minimum value and the second minimum value as the two scale mark numbers closest to the straight line where the pointer is located.
In step S4, the areas where the two scale marks are located are respectively cut out from the picture of the dial area of the meter to be identified.
In step S5, the numerical values of the two scale mark numbers are identified by the third depth network model, and the reading is performed by the angle method.
Specifically, the areas where the two cut-out scale line numbers are located are respectively subjected to slope correction by using a contour method and then are divided into digital characters by using a projection method, the divided digital characters are sequentially input into a third depth network model so as to identify the numerical values of the two scale line numbers, and the reading of the instrument is calculated by using an angle method.
Specifically, in another embodiment, when the two scale line numbers are identified, if the numerical values of the two scale line numbers closest to the straight line where the pointer is located cannot be identified, it is determined that the two scale line numbers are blocked by the pointer of the meter to be identified, and then, the area of the two scale line numbers closest to the straight line where the pointer is located in the remaining scale line numbers except the two scale line numbers in the dial plate of the meter to be identified is confirmed; respectively intercepting the areas of two scale mark numbers which are closest to the straight line where the pointer is located in the rest of scale mark numbers from the intercepted picture of the dial plate area of the instrument to be identified; respectively carrying out inclination correction on the areas of two scale mark numbers which are closest to the straight line where the pointer is located in the other cut scale mark numbers by using a contour method, then dividing the areas into digital characters by using a projection method, and sequentially inputting the divided digital characters into a third depth network model so as to identify the numerical values of the two scale mark numbers which are closest to the straight line where the pointer is located in the other scale mark numbers; and determining the numerical values of the two scale mark numbers closest to the straight line of the pointer according to the numerical values of the two scale mark numbers closest to the straight line of the pointer in the rest scale mark numbers.
In another embodiment, if one of the numerical values of the two scale mark numbers closest to the straight line where the pointer is located cannot be identified, the scale mark number which cannot be identified in the two scale mark numbers is judged to be blocked by the pointer of the meter to be identified, and at this time, the area of one scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers except the two scale mark numbers in the dial plate of the meter to be identified is confirmed; intercepting the area of one scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers from the intercepted picture of the dial plate area of the instrument to be identified; the area of one of the cut-off rest scale mark numbers which is closest to the straight line where the pointer is located is subjected to slope correction by using a contour method and then is divided into digital characters by using a projection method, and the divided digital characters are sequentially input into a third depth network model, so that the numerical value of one of the rest scale mark numbers which is closest to the straight line where the pointer is located is identified; and determining the numerical values of the two scale mark numbers closest to the straight line of the pointer according to the numerical value of the scale mark number closest to the straight line of the pointer in the rest scale mark numbers and the numerical value of the previously identified scale mark number.
Wherein, adopting the angle method to calculate the reading of this instrument includes: respectively connecting two scale mark numbers closest to the straight line where the pointer is located with the center of a circle of a dial plate of the instrument to be identified so as to obtain a first straight line and a second straight line; determining a first included angle theta 1 between the first straight line and the straight line where the pointer is located, and determining a second included angle theta 2 between the second straight line and the straight line where the pointer is located; and determining the reading v of the meter to be identified according to an angle method.
Wherein, the angle method is as follows:
where v1 is the value of the smaller of the two tick mark numbers closest to the line on which the pointer is located; v2 is the value of the larger of the two tick mark numbers closest to the line on which the pointer is located.
In this embodiment, the pointer instrument reading identification method further includes building a first depth network model, building a second depth network model, and building a third depth network model.
Constructing the first deep network model comprises: collecting a picture set of each instrument in a monitoring range through a visible light camera, wherein the picture set comprises pictures of each instrument shot under a plurality of shooting angles, a plurality of lighting conditions (light, dark and reflective lamps), a plurality of backgrounds and a plurality of pointer rotation angles; marking the dial plate area and the type of each instrument in the picture set, and recording the position information of the dial plate area of each instrument and the position information of the scale mark digital area of each instrument; and training the marked picture set through a target detection network model to obtain a first depth network model. Specifically, when position information of a dial and a scale mark number is recorded in an XML (extensible markup language) file and the position information is extracted by parsing the XML tag file.
The instrument types include various pointer instruments such as a thermometer, a hygrometer, a voltmeter and an ammeter. The target detection network model comprises: and target detection network models such as YOLO v3, RCNN series, Mask-RCNN, R-FCN, YOLO, SSD, FPN and the like.
Constructing the second deep network model comprises: analyzing the position information of the dial area of each instrument recorded in the file, and intercepting the dial area of each instrument from the picture of each instrument to obtain a dial picture set; marking a half pointer area, a dial circle center area and a scale mark number area of each dial in a dial picture set, wherein the half pointer area is a rectangular frame, and the half pointer is marked on a diagonal line of the rectangular frame; and training the marked dial plate picture set through the target detection network model so as to obtain a second depth network model. Specifically, in the embodiment, the half pointer of each dial, the center of the dial, and the optimal labeling area of the scale mark number are determined according to the K-means algorithm, so that the positioning accuracy is improved. Fig. 2 is a view of the dial plate cut out from the instrument picture, which is easy to understand. Fig. 3 shows the marked half pointer area (as indicated by a), the tick mark number area (as indicated by B), and the dial centre area (as indicated by C).
Constructing the third deep network model comprises: analyzing the position information of the scale mark digital area of each instrument recorded in the file, and intercepting the scale mark digital area of each instrument from the picture of each dial plate to obtain a scale mark digital picture set; adopting a contour method to perform inclination correction on each scale mark digit of the scale mark digit picture set, and segmenting each corrected scale mark digit by a projection method to obtain a digit character picture set; and training the digital character picture set by adopting the CaffeNet network model migration learning MNIST data set weight so as to obtain a third deep network model.
In summary, according to the deep learning-based pointer instrument reading identification method in the embodiment, the instrument type and dial position information of the instrument to be identified are obtained through the first depth network model, the digital position information on the half pointer, the dial center and the scale mark is obtained through the second depth network model, the digital identification on the dial scale mark is realized through the third depth network model, the reading of the pointer instrument is realized through training the three models, the reading efficiency is higher, and various types of pointer instruments can be identified; in addition, in the process of establishing the model, the sampled picture sample data are pictures with different shooting angles, different illumination conditions, different backgrounds and different types, so that the method can accurately read the instrument under the conditions of complex background, uneven illumination, image blurring, mirror reflection, instrument inclination and the like, and the method has good robustness and high universality; meanwhile, compared with the angle method reading in the prior art, the angle method reading in the embodiment omits the step of instrument inclination correction, effectively improves the speed of instrument positioning and dial element extraction, improves the reading precision, and compared with the method for identifying the instrument reading by adopting image processing in the prior art, omits the step of image preprocessing, and further improves the speed of intelligent instrument reading.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.
Claims (10)
1. A reading identification method of a pointer instrument based on deep learning is used for reading identification of the pointer instrument, and is characterized by comprising the following steps:
inputting a picture of the instrument to be recognized into a first depth network model so as to obtain the type of the instrument to be recognized and the area coordinate of the dial plate of the instrument to be recognized in the picture;
the area where the dial plate of the instrument to be identified is located is cut out from the picture;
inputting the intercepted picture of the dial plate area of the instrument to be recognized into a second depth network model so as to obtain the circle center coordinate of the dial plate of the instrument to be recognized and the coordinates of each scale mark number, determining the coordinate of the half pointer point of the dial plate of the instrument to be recognized and the straight line where the pointer is located, and determining the area of two scale mark numbers closest to the straight line where the pointer is located;
respectively intercepting the areas where the two scale mark numbers are located from the intercepted picture of the dial plate area of the instrument to be identified;
the areas where the two cut scale mark numbers are located are respectively subjected to slope correction by a contour method and then are divided into digital characters by a projection method, and the divided digital characters are sequentially input into a third depth network model so as to identify the numerical values of the two scale mark numbers closest to the straight line where the pointer is located; and
and calculating the reading of the meter to be identified by adopting an angle method, and acquiring the scale unit of the meter to be identified according to the type of the meter to be identified.
2. The deep learning-based pointer instrument reading identification method of claim 1, further comprising constructing the first depth network model, wherein constructing the first depth network model comprises:
collecting a picture set of each instrument in a monitoring range through a visible light camera, wherein the picture set comprises pictures of each instrument shot under a plurality of shooting angles, a plurality of lighting conditions, a plurality of backgrounds and a plurality of pointer rotation angles;
marking the dial plate area and the type of each instrument in the picture set, and recording the position information of the dial plate area of each instrument and the position information of the scale mark digital area of each instrument; and
and training the marked picture set through a target detection network model to obtain the first deep network model.
3. The deep learning-based pointer instrument reading identification method of claim 2, further comprising constructing the second depth network model, wherein the constructing the second depth network model comprises:
according to the position information of the dial area of each instrument, the dial area of each instrument is cut out from the picture of each instrument so as to obtain a dial picture set;
marking a half pointer area, a dial circle center area and a scale mark number area of each dial in the dial picture set, wherein the half pointer area is a rectangular frame, and the half pointer is marked on a diagonal line of the rectangular frame; and
and training the marked dial plate picture set through the target detection network model so as to obtain the second deep network model.
4. The deep learning-based pointer instrument reading identification method of claim 3, wherein labeling a half pointer area, a dial circle center area and a scale mark number area of each dial in the dial picture set comprises:
and determining the half pointer, the dial center and the scale mark area of the scale mark number of each dial according to the K mean value algorithm.
5. The deep learning-based pointer instrument reading identification method of claim 3, further comprising constructing the third depth network model, wherein constructing the third depth network model comprises:
according to the position information of the scale mark digital areas of the instruments, the scale mark digital areas of the instruments are intercepted from the pictures of the dials to obtain a scale mark digital picture set;
adopting a contour method to perform inclination correction on each scale mark digit of the scale mark digit picture set, and segmenting each corrected scale mark digit by a projection method to obtain a digit character picture set; and
and training the digital character picture set by adopting CaffeNet network model migration learning MNIST data set weight so as to obtain the third deep network model.
6. The deep learning-based pointer instrument reading identification method of claim 3, wherein the determining coordinates of the half pointer tip of the instrument dial comprises:
determining Euclidean distances from the center coordinates of the dial plate of the instrument to be identified to four vertexes of the semi-pointer rectangular frame; and
and selecting the vertex coordinate corresponding to the maximum Euclidean distance as the coordinate of the half pointer tip.
7. The deep learning-based pointer instrument reading identification method of claim 1, wherein the determining the area of two tick mark numbers closest to the straight line on which the pointer is located comprises:
determining Euclidean distances from coordinates of each scale mark number of the instrument to be identified to coordinates of the needle point of the half pointer; and
and selecting the scale mark numbers corresponding to the Euclidean distance minimum value and the second minimum value as the areas of the two scale mark numbers closest to the straight line where the pointer is located.
8. The method for recognizing the reading of the pointer instrument based on the deep learning as claimed in claim 1, wherein the calculating the reading of the instrument to be recognized by adopting the angle method comprises:
respectively connecting two scale mark numbers closest to the straight line where the pointer is located with the center of a circle of the dial plate of the instrument to be identified so as to obtain a first straight line and a second straight line;
determining a first included angle theta 1 between the first straight line and the straight line where the pointer is located, and determining a second included angle theta 2 between the second straight line and the straight line where the pointer is located; and
determining the reading v of the meter to be identified according to an angle method,
wherein, the angle method is as follows:
wherein v1 is the value of the smaller of the two tick mark numbers closest to the line on which the pointer lies; v2 is the value of the larger of the two tick mark numbers closest to the line on which the pointer lies.
9. The method for recognizing the reading of the pointer instrument based on the deep learning as claimed in claim 1, wherein the step of dividing the area where the two cut-off scale line numbers are located into the number characters by a projection method after the two cut-off scale line numbers are respectively subjected to the slope correction by a contour method, and sequentially inputting the divided number characters into the third depth network model so as to recognize the numerical values of the two scale line numbers closest to the straight line where the pointer is located comprises the steps of:
if the numerical values of the two scale mark numbers closest to the straight line where the pointer is located cannot be identified, judging that the two scale mark numbers are shielded by the pointer of the instrument to be identified, and then confirming the areas of the two scale mark numbers closest to the straight line where the pointer is located in the rest scale mark numbers except the two scale mark numbers in the dial plate of the instrument to be identified;
respectively intercepting the areas of two scale mark numbers which are closest to the straight line where the pointer is located in the rest of scale mark numbers from the intercepted picture of the dial plate area of the instrument to be identified;
respectively carrying out inclination correction on the areas of the two scale mark numbers which are closest to the straight line where the pointer is located in the intercepted rest scale mark numbers by using a contour method, then segmenting the areas into digital characters by using a projection method, and sequentially inputting the segmented digital characters into a third depth network model so as to identify the numerical values of the two scale mark numbers which are closest to the straight line where the pointer is located in the rest scale mark numbers; and
and determining the numerical values of the two scale mark numbers closest to the straight line of the pointer according to the numerical values of the two scale mark numbers closest to the straight line of the pointer in the rest scale mark numbers.
10. The method for recognizing the reading of the pointer instrument based on the deep learning as claimed in claim 1, wherein the step of dividing the area where the two cut-off scale line numbers are located into the number characters by a projection method after the two cut-off scale line numbers are respectively subjected to the slope correction by a contour method, and sequentially inputting the divided number characters into the third depth network model so as to recognize the numerical values of the two scale line numbers closest to the straight line where the pointer is located comprises the steps of:
if one of the numerical values of the two scale mark numbers closest to the straight line where the pointer is located cannot be identified, judging that the scale mark number which cannot be identified in the two scale mark numbers is shielded by the pointer of the instrument to be identified, and then confirming the area of one scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers except the two scale mark numbers in the dial plate of the instrument to be identified;
intercepting the area of one graduation mark number closest to the straight line where the pointer is located in the rest graduation mark numbers from the intercepted picture of the dial plate area of the instrument to be identified;
the area of one of the cut-off scale mark numbers which is closest to the straight line where the pointer is located is subjected to slope correction by using a contour method and then is divided into digital characters by using a projection method, and the divided digital characters are sequentially input into a third depth network model, so that the numerical value of one of the cut-off scale mark numbers which is closest to the straight line where the pointer is located is identified; and
and determining the numerical values of the two scale mark numbers closest to the straight line where the pointer is located according to the numerical value of the scale mark number closest to the straight line where the pointer is located in the rest scale mark numbers and the numerical value of the previously identified scale mark number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910894088.9A CN110659636B (en) | 2019-09-20 | 2019-09-20 | Pointer instrument reading identification method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910894088.9A CN110659636B (en) | 2019-09-20 | 2019-09-20 | Pointer instrument reading identification method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110659636A true CN110659636A (en) | 2020-01-07 |
| CN110659636B CN110659636B (en) | 2022-05-13 |
Family
ID=69038276
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910894088.9A Active CN110659636B (en) | 2019-09-20 | 2019-09-20 | Pointer instrument reading identification method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110659636B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291691A (en) * | 2020-02-17 | 2020-06-16 | 合肥工业大学 | Deep learning-based substation secondary equipment instrument panel reading detection method |
| CN111368906A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Pointer type oil level indicator reading identification method based on deep learning |
| CN111950369A (en) * | 2020-07-10 | 2020-11-17 | 济南信通达电气科技有限公司 | Transformer substation pointer type disconnecting link action degree identification method and device |
| CN112115895A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Pointer type instrument reading identification method and device, computer equipment and storage medium |
| CN112183530A (en) * | 2020-09-24 | 2021-01-05 | 上海三高计算机中心股份有限公司 | Meter reading identification system |
| CN112308054A (en) * | 2020-12-29 | 2021-02-02 | 广东科凯达智能机器人有限公司 | Automatic reading method of multifunctional digital meter based on target detection algorithm |
| CN112489071A (en) * | 2020-11-03 | 2021-03-12 | 辽宁长江智能科技股份有限公司 | Pointer water meter identification method and system |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112560839A (en) * | 2020-12-02 | 2021-03-26 | 随锐科技集团股份有限公司 | Automatic identification method and system for reading of pointer instrument |
| CN113029220A (en) * | 2021-01-28 | 2021-06-25 | 开放智能机器（上海）有限公司 | State recognition system and method for industrial instrument panel |
| CN113312941A (en) * | 2020-02-26 | 2021-08-27 | 北京同邦卓益科技有限公司 | Method and device for monitoring instrument panel |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113566863A (en) * | 2021-06-11 | 2021-10-29 | 北京眸视科技有限公司 | Reading method and device of pointer table |
| CN113837159A (en) * | 2021-11-26 | 2021-12-24 | 中化学交通建设集团有限公司 | Instrument reading identification method and device based on machine vision |
| CN113837166A (en) * | 2021-08-20 | 2021-12-24 | 北京工业大学 | Pointer type instrument automatic reading method based on deep learning |
| CN114549814A (en) * | 2022-01-25 | 2022-05-27 | 福建和盛高科技产业有限公司 | Instrument reading identification method based on yolox detection |
| CN115690771A (en) * | 2022-12-29 | 2023-02-03 | 常州海图信息科技股份有限公司 | Electric power room instrument inspection method and system |
| WO2024141157A1 (en) * | 2022-12-27 | 2024-07-04 | Siemens Aktiengesellschaft | System, apparatus and method for processing data from one or more physical devices monitoring one or more assets in a facility |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070211239A1 (en) * | 2006-03-08 | 2007-09-13 | Electronic Scripting Products, Inc. | Optical navigation apparatus using fixed beacons and a centroid sensing device |
| CN103207987A (en) * | 2013-02-28 | 2013-07-17 | 华北电力大学 | Indicating value identification method of dial instrument |
| CN105809179A (en) * | 2014-12-31 | 2016-07-27 | 中国科学院深圳先进技术研究院 | Pointer type instrument reading recognition method and device |
| CN108460327A (en) * | 2018-01-12 | 2018-08-28 | 河南大学 | A kind of readings of pointer type meters automatic identifying method based on image procossing |
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机系统工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| US20190095739A1 (en) * | 2017-09-27 | 2019-03-28 | Harbin Institute Of Technology | Adaptive Auto Meter Detection Method based on Character Segmentation and Cascade Classifier |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
-
2019
- 2019-09-20 CN CN201910894088.9A patent/CN110659636B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070211239A1 (en) * | 2006-03-08 | 2007-09-13 | Electronic Scripting Products, Inc. | Optical navigation apparatus using fixed beacons and a centroid sensing device |
| CN103207987A (en) * | 2013-02-28 | 2013-07-17 | 华北电力大学 | Indicating value identification method of dial instrument |
| CN105809179A (en) * | 2014-12-31 | 2016-07-27 | 中国科学院深圳先进技术研究院 | Pointer type instrument reading recognition method and device |
| US20190095739A1 (en) * | 2017-09-27 | 2019-03-28 | Harbin Institute Of Technology | Adaptive Auto Meter Detection Method based on Character Segmentation and Cascade Classifier |
| CN108460327A (en) * | 2018-01-12 | 2018-08-28 | 河南大学 | A kind of readings of pointer type meters automatic identifying method based on image procossing |
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机系统工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
Non-Patent Citations (2)
| Title |
|---|
| TENGTENG ZHANG ET AL.: "Automatic Recognition of Instrument Dial Readings with Multiple Reference Points", 《2017 CHINESE AUTOMATION CONGRESS CAC》 * |
| 魏琦 等: "仪表自动判读方法的研究与实现", 《信息技术》 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291691A (en) * | 2020-02-17 | 2020-06-16 | 合肥工业大学 | Deep learning-based substation secondary equipment instrument panel reading detection method |
| CN113312941A (en) * | 2020-02-26 | 2021-08-27 | 北京同邦卓益科技有限公司 | Method and device for monitoring instrument panel |
| CN111368906B (en) * | 2020-02-29 | 2023-05-23 | 华南理工大学 | Pointer type oil level meter reading identification method based on deep learning |
| CN111368906A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Pointer type oil level indicator reading identification method based on deep learning |
| CN111950369A (en) * | 2020-07-10 | 2020-11-17 | 济南信通达电气科技有限公司 | Transformer substation pointer type disconnecting link action degree identification method and device |
| CN112115895A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Pointer type instrument reading identification method and device, computer equipment and storage medium |
| CN112183530A (en) * | 2020-09-24 | 2021-01-05 | 上海三高计算机中心股份有限公司 | Meter reading identification system |
| CN112115895B (en) * | 2020-09-24 | 2023-12-22 | 深圳市赛为智能股份有限公司 | Pointer type instrument reading identification method, pointer type instrument reading identification device, computer equipment and storage medium |
| CN112489071A (en) * | 2020-11-03 | 2021-03-12 | 辽宁长江智能科技股份有限公司 | Pointer water meter identification method and system |
| CN112507815A (en) * | 2020-11-24 | 2021-03-16 | 北京超维世纪科技有限公司 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
| CN112560839A (en) * | 2020-12-02 | 2021-03-26 | 随锐科技集团股份有限公司 | Automatic identification method and system for reading of pointer instrument |
| CN112308054A (en) * | 2020-12-29 | 2021-02-02 | 广东科凯达智能机器人有限公司 | Automatic reading method of multifunctional digital meter based on target detection algorithm |
| CN113029220A (en) * | 2021-01-28 | 2021-06-25 | 开放智能机器（上海）有限公司 | State recognition system and method for industrial instrument panel |
| CN113566863A (en) * | 2021-06-11 | 2021-10-29 | 北京眸视科技有限公司 | Reading method and device of pointer table |
| CN113566863B (en) * | 2021-06-11 | 2023-12-26 | 北京眸视科技有限公司 | Pointer table reading method and device |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113469178B (en) * | 2021-07-05 | 2024-03-01 | 安徽南瑞继远电网技术有限公司 | Power meter identification method based on deep learning |
| CN113837166A (en) * | 2021-08-20 | 2021-12-24 | 北京工业大学 | Pointer type instrument automatic reading method based on deep learning |
| CN113837166B (en) * | 2021-08-20 | 2024-03-12 | 北京工业大学 | Automatic pointer instrument reading method based on deep learning |
| CN113837159A (en) * | 2021-11-26 | 2021-12-24 | 中化学交通建设集团有限公司 | Instrument reading identification method and device based on machine vision |
| CN114549814A (en) * | 2022-01-25 | 2022-05-27 | 福建和盛高科技产业有限公司 | Instrument reading identification method based on yolox detection |
| WO2024141157A1 (en) * | 2022-12-27 | 2024-07-04 | Siemens Aktiengesellschaft | System, apparatus and method for processing data from one or more physical devices monitoring one or more assets in a facility |
| CN115690771A (en) * | 2022-12-29 | 2023-02-03 | 常州海图信息科技股份有限公司 | Electric power room instrument inspection method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110659636B (en) | 2022-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110659636B (en) | Pointer instrument reading identification method based on deep learning | |
| CN112257676B (en) | Pointer type instrument reading method and system and inspection robot | |
| CN112818988B (en) | Automatic identification reading method and system for pointer instrument | |
| CN110580480B (en) | Meter reading identification method based on image processing | |
| Chi et al. | Machine vision based automatic detection method of indicating values of a pointer gauge | |
| CN105091922B (en) | A kind of pointer gauge Recognition of Reading method based on virtual dial plate | |
| CN112149667B (en) | Automatic reading method of pointer instrument based on deep learning | |
| CN111950330A (en) | Pointer instrument indicating number detection method based on target detection | |
| CN103994786B (en) | Image detecting method for arc ruler lines of pointer instrument scale | |
| CN111368906B (en) | Pointer type oil level meter reading identification method based on deep learning | |
| CN104657711B (en) | A kind of readings of pointer type meters automatic identifying method of robust | |
| CN110852954B (en) | Image inclination correction method and system for pointer instrument | |
| CN108182433A (en) | A kind of meter reading recognition methods and system | |
| CN107239742A (en) | A kind of gauge pointer scale value calculating method | |
| CN105740856A (en) | Method for reading readings of pointer instrument based on machine vision | |
| CN109508709B (en) | Single pointer instrument reading method based on machine vision | |
| CN109448043A (en) | Standing tree height extracting method under plane restriction | |
| CN112801094A (en) | Pointer instrument image inclination correction method | |
| CN108447092B (en) | Method and device for visually positioning marker | |
| CN114663744A (en) | Instrument automatic identification method and system based on machine learning | |
| CN105423975A (en) | Calibration system and method of large-size workpiece | |
| CN110211178A (en) | A kind of pointer instrument recognition methods calculated using projection | |
| CN113837159A (en) | Instrument reading identification method and device based on machine vision | |
| Yang et al. | Auto-recognition Method for Pointer-type Meter Based on Binocular Vision. | |
| CN112560839A (en) | Automatic identification method and system for reading of pointer instrument |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200310 Address after: 100192 Beijing city Haidian District Baosheng Road No. 1 north of the Austrian Science Park Building No. 19 Beijing Suirui Center Applicant after: Suirui Technology Group Co., Ltd Applicant after: Jiangxi Smart Technology Co., Ltd. Address before: 100192 Beijing city Haidian District Baosheng Road No. 1 north of the Austrian Science Park Building No. 19 Beijing Suirui Center Applicant before: Suirui Technology Group Co., Ltd |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |