CN114882487A - Pointer instrument reading method and system - Google Patents
Pointer instrument reading method and system Download PDFInfo
- Publication number
- CN114882487A CN114882487A CN202210495907.4A CN202210495907A CN114882487A CN 114882487 A CN114882487 A CN 114882487A CN 202210495907 A CN202210495907 A CN 202210495907A CN 114882487 A CN114882487 A CN 114882487A
- Authority
- CN
- China
- Prior art keywords
- pointer
- scale
- instrument
- scale number
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Abstract
The invention provides a reading method and a system for indicating numbers of a pointer instrument, wherein the reading method processes an image of the pointer instrument to be detected to obtain a front view dial plate, a central point of a pointer rotating shaft, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale number key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front-view dial plate, and calculating to obtain the current instrument reading of the pointer instrument through a local angle method according to an included angle formed by the central point of a pointer rotating shaft and a connecting line between the two scale number key points, two sub-included angles formed by the included angle divided by the pointer direction line and scale numbers corresponding to the two scale number key points. The key information for calculating the instrument readings can be obtained from the current instrument image, a large amount of labor and time can be saved, and the method and the device can be suitable for instruments with uniform scales and non-uniform scales.
Description
Technical Field
The invention relates to the technical field of pointer instruments, in particular to a pointer instrument reading method and a pointer instrument reading system.
Background
Pointer instruments are widely applied in daily life and industrial fields, and how to automatically read the readings of the instruments efficiently and accurately is very important. The reading of the instrument readings mainly comprises four steps of instrument detection, dial plate correction, pointer positioning and reading calculation, and the realization of an efficient, stable and widely applicable instrument reading method is still difficult due to the influence of the environment of the instrument, the pointer shadow and the interference of irrelevant information on an instrument image.
Currently, reading of meter readings is mainly performed by a related technology of deep learning. The method based on deep learning can be suitable for various instruments under different scenes and has high stability and instantaneity. However, the deep learning-based method requires a large number of data sets for training, and the existing public data sets have no data sets specific to meters. This type of method therefore requires a significant amount of time to produce the data tag. In addition, the existing method has the problem that the meter reading is difficult to obtain only from the meter to be read currently. For example, in the dial plate correction process, a front view instrument of the same type is often required to be used as a template, and correction is performed in a characteristic point matching mode. In the process of reading number, the range of the instrument needs to be manually input, the number of the current instrument cannot be automatically calculated, and the existing reading method for the number is usually suitable for the instrument with uniform scale and is not suitable for the instrument with non-uniform scale.
Disclosure of Invention
The invention provides a reading method and a reading system for the number of a pointer instrument, which can acquire key information required by calculating the number of the instrument from a current instrument image and realize the automatic reading of the number of the instrument with uniform scale and non-uniform scale; and does not require a large number of precise annotations when making the data set.
In a first aspect, the present invention provides a pointer instrument reading method, including: acquiring an image of a pointer instrument to be detected; obtaining a front view dial, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected; detecting two scale digital key points which are respectively arranged on two sides of a pointer pointing line and are closest to the front end of the pointer pointing line on the front-view dial plate; detecting the included angles formed by the central point of the pointer rotating shaft and the connecting line between the two scale number key points respectively; detecting two sub-included angles formed by dividing the included angle by a pointer pointing line; and calculating the current instrument reading of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
In the scheme, the image of the pointer instrument to be detected is processed to obtain a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale number key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front-view dial plate, and calculating to obtain the current instrument reading of the pointer instrument through a local angle method according to an included angle formed by the central point of a pointer rotating shaft and a connecting line between the two scale number key points, two sub-included angles formed by the included angle divided by the pointer direction line and scale numbers corresponding to the two scale number key points. The method does not need to take an orthophoria instrument of the same type as the prior art as a template, then carries out correction in a characteristic point matching mode, and carries out the number indicating calculation by manually inputting the measuring range of the instrument. The method disclosed by the application can acquire the key information for calculating the instrument readings from the current instrument image without prior information, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved. In addition, the key points of the scale numbers and the scale numbers on the left side and the right side of the pointer direction line are identified, and then the meter readings are calculated by using a local angle method, so that the method and the device are suitable for meters with uniform scales and non-uniform scales.
In a specific embodiment, obtaining the front view dial, the pointer rotation shaft center point, the pointer direction line, the scale number and the corresponding scale number key point according to the pointer instrument image to be measured includes: detecting a dial plate, a pointer rotating shaft central point, scale numbers and corresponding scale number key points in an image of a pointer instrument to be detected; carrying out ellipse fitting on the scale number key points to obtain a fitting ellipse; and obtaining the front view dial according to the central point of the pointer rotating shaft, the fitting ellipse and the scale number key point. The calibration of the pointer instrument image to be measured can be realized by using the scale number key points on the current instrument image and the center point of the pointer rotating shaft. The method does not need to use a same type of front view instrument as a template and then carry out correction in a characteristic point matching mode like the prior art, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved.
In a specific embodiment, obtaining the front view dial according to the central point of the rotating shaft of the pointer, the fitting ellipse and the key point of the scale number comprises: projecting all points on the dial plate in the pointer instrument image to be detected onto a hypothetical dial plate plane by adopting a perspective transformation correction algorithm according to the central point of the pointer rotating shaft and the fitting ellipse; and processing the dial projected on the assumed dial plane by using the symmetry of the scale digital key points and adopting a rotation correction algorithm to obtain the front view dial. The key information on the current instrument image is conveniently utilized to quickly correct the pointer instrument image to be detected to obtain the front view dial plate.
In a specific embodiment, obtaining the front view dial, the pointer rotation shaft center point, the pointer direction line, the scale number and the corresponding scale number key point according to the pointer instrument image to be measured further includes: detecting two end points on a pointer main body on a front view dial plate; obtaining the middle point of the pointer trunk according to the two end points; and determining the direction of the pointer and a pointer direction line according to the central point of the pointer rotating shaft and the central point of the pointer trunk. The pointer direction and the pointer direction line can be quickly obtained by using the pointer key point information on the current instrument image.
In one specific embodiment, determining the pointing direction of the pointer and the pointer pointing line according to the center point of the pointer rotation shaft and the center point of the pointer trunk comprises: establishing a rectangular coordinate system on the front view dial plate; detecting coordinates of the central point of the pointer rotating shaft and the central point of the pointer trunk on a rectangular coordinate system; determining the direction of the pointer according to the coordinates of the central point of the pointer rotating shaft and the central point of the pointer trunk; and taking a connecting line of the central point of the pointer rotating shaft and the front end point of the pointer trunk as a pointer pointing line. The point of the pointer can be quickly and accurately determined by using the horizontal and vertical coordinate difference of the central point of the pointer rotating shaft and the central point of the pointer trunk under the rectangular coordinate system.
In a specific embodiment, establishing a rectangular coordinate system on the face plate includes: and establishing an x-y rectangular coordinate system on the front-view dial plate by taking the central point of the pointer rotating shaft as an original point and taking the parallel direction of key point connecting lines of scale numbers corresponding to the minimum scale number and the maximum scale number on the front-view dial plate as an x axis. The right-angle coordinate line can be conveniently established on the front-view dial plate, and the pointing direction of the pointer can be conveniently and rapidly determined according to the horizontal and vertical coordinate difference between the central point of the pointer rotating shaft and the central point of the pointer trunk.
In a specific embodiment, the scale numbers corresponding to the two scale number key points are respectively: a first scale number, and a second scale number greater than the first scale number. According to the included angle, the two sub-included angles and the two scale numbers, calculating the current instrument reading number of the pointer instrument comprises the following steps: calculating the current instrument reading value of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers by adopting the following formula:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Representing a first scale number; v 2 Representing a second scale number; alpha represents an included angle; alpha is alpha 1 And the scale number key point corresponding to the first scale number in the two sub-included angles is positioned in the sub-included angle on the same side of the pointer pointing line. The current meter reading of the pointer meter is calculated by a local angle method quickly and accurately.
In a second aspect, the present invention also provides a pointer instrument reading system, comprising: the device comprises an image acquisition module, a detection and dial plate correction module, a scale number key point identification module, an included angle identification module and a number indication calculation module. The image acquisition module is used for acquiring an image of the pointer instrument to be detected. The detection and dial plate correction module is used for obtaining a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected. The scale number key point identification module is used for detecting two scale number key points which are arranged on the front-view dial plate and are respectively arranged on two sides of the pointer direction line and are closest to the front end of the pointer direction line. The included angle identification module is used for detecting included angles formed by the central point of the pointer rotating shaft and a connecting line between the central point and the two scale number key points respectively; and the device is also used for detecting two sub-included angles formed by dividing the included angle by the pointer pointing line. And the number-indicating calculation module is used for calculating the current instrument number of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
In the scheme, the image of the pointer instrument to be detected is processed to obtain a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale digital key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front view dial plate, and calculating the current instrument reading number of the pointer instrument by a local angle method according to an included angle formed by connecting the central point of the pointer rotating shaft with the connecting line between the two scale digital key points, two sub-included angles formed by dividing the included angle by the pointer direction line and scale numbers corresponding to the two scale digital key points. The method does not need to take a same type of front-view instrument as a template, then carries out correction in a characteristic point matching mode, and carries out the reading calculation by manually inputting the measuring range of the instrument. The method disclosed by the application can acquire the key information for calculating the instrument indication number from the current instrument image without prior information, so that a large amount of accurate labeling is not needed when a data set is manufactured, and a large amount of labor and time can be saved. In addition, the key points of the scale numbers and the scale numbers on the left side and the right side of the pointer direction line are identified, and then the meter readings are calculated by using a local angle method, so that the method and the device are suitable for meters with uniform scales and non-uniform scales.
In a particular embodiment, the detection and dial correction module comprises: detection module and dial plate correction module. The detection module is used for positioning the pointer instrument image to be detected based on the improved YOLOv4 model, and detecting a pointer rotating shaft central point, an end point on a pointer main body, scale numbers and corresponding scale number key points; the improved YOLOv4 model consists of a CSPDarknet backbone network, SPP space pyramid pooling, a PANet multi-scale feature extraction module and a YOLO Head output layer; the standard convolution in the CSPDarknet backbone network is replaced by a deep separable convolution. And the dial plate correction module is used for obtaining a front view dial plate according to the pointer instrument image to be detected. The parameter calculation amount can be reduced to a greater extent, and the target detection speed is improved; meanwhile, richer characteristic information can be obtained, and the capability of the network for detecting the small target is improved.
In a specific embodiment, the scale numbers corresponding to the two scale number key points are respectively: a first scale number, and a second scale number greater than the first scale number. The number calculation module calculates the current meter number value of the pointer meter according to the included angle, the two sub-included angles and the two scale numbers by adopting the following formula:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Representing a first scale number; v 2 Representing a second scale number; alpha represents an included angle; alpha is alpha 1 And the scale number key point corresponding to the first scale number in the two sub-included angles is positioned in the sub-included angle on the same side of the pointer pointing line. The current meter reading of the pointer meter is calculated and obtained quickly and accurately through a local angle method.
Drawings
FIG. 1 is a flowchart of a method for reading an indication of a pointer instrument according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a rectangular coordinate system established on a front view dial according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a partial angle method on a front view dial according to an embodiment of the present invention;
FIG. 4 is a flow chart of a training and testing phase of a pointer reading system according to an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a pointer reading system according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of another pointer instrument reading system provided by an embodiment of the present invention;
FIG. 7 is a diagram of a depth separable convolution structure according to an embodiment of the present invention;
fig. 8 is a diagram illustrating an improved YOLOv4 network structure according to an example of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to facilitate understanding of the pointer and meter reading method provided by the embodiment of the present invention, an application scenario of the pointer and meter reading method provided by the embodiment of the present invention is first described below, where the pointer and meter reading method is applied to a pointer and meter reading process. The pointer instrument can be various pointer instruments such as an ammeter, a voltmeter, a resistance meter and a flow meter. In addition, the pointer instrument can be a uniform-scale instrument or a non-uniform-scale instrument. The reading method of the pointer instrument will be described in detail with reference to the accompanying drawings.
Referring to fig. 1, a method for reading a pointer instrument reading according to an embodiment of the present invention includes:
step 10: acquiring an image of a pointer instrument to be detected;
step 20: obtaining a front view dial, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected;
step 30: detecting two scale digital key points which are respectively arranged on two sides of a pointer pointing line and are closest to the front end of the pointer pointing line on the front-view dial plate;
step 40: detecting the included angles formed by the central point of the pointer rotating shaft and the connecting line between the two scale number key points respectively;
step 50: detecting two sub-included angles formed by dividing the included angle by a pointer pointing line;
step 60: and calculating the current instrument reading of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
In the scheme, the image of the pointer instrument to be measured is processed to obtain a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale number key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front-view dial plate, and calculating to obtain the current instrument reading of the pointer instrument through a local angle method according to an included angle formed by the central point of a pointer rotating shaft and a connecting line between the two scale number key points, two sub-included angles formed by the included angle divided by the pointer direction line and scale numbers corresponding to the two scale number key points. The method does not need to take an orthophoria instrument of the same type as the prior art as a template, then carries out correction in a characteristic point matching mode, and carries out the number indicating calculation by manually inputting the measuring range of the instrument. The method disclosed by the application can acquire the key information for calculating the instrument readings from the current instrument image without prior information, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved. In addition, the key points of the scale numbers and the scale numbers on the left side and the right side of the pointer direction line are identified, and then the meter readings are calculated by using a local angle method, so that the method and the device are suitable for meters with uniform scales and non-uniform scales. The above steps will be described in detail with reference to the accompanying drawings.
Referring to fig. 1 and 4, first, an image of a pointer instrument to be measured is obtained, where the image of the pointer instrument to be measured at least includes a dial image of the instrument, and the dial image includes images of a pointer, scale numbers, scale lines, and the like. And pointing the pointer in the pointer instrument to be measured to indicate the instrument display number when the acquired pointer instrument image to be measured is acquired.
Next, with continued reference to fig. 1, according to the pointer instrument image to be measured, a front view dial, a pointer rotation shaft center point, a pointer direction line, a scale number, and a corresponding scale number key point are obtained, which are the basis of the subsequent link. Namely, in the step, the dial in the input pointer instrument image to be measured is mainly and completely positioned, and the dial with distortion is corrected to be a front view dial. The center point of the pointer rotating shaft and the pointer pointing line are also detected. It should be noted that the pointer pointing line is an aggregate of the pointing direction of the pointer and the central axis of the pointer, and the central point of the rotating shaft of the pointer is necessarily located on the pointer pointing line or on the extension line of the pointer pointing line. I.e. the pointer to the line contains not only direction information but also position information. Meanwhile, in this step, the scale numbers and the scale number key points corresponding to each scale number need to be detected. It should be explained that the scale number refers to a number which is provided on the dial of the instrument and indicates the size degree of a certain scale mark, and the key point of the scale number is a characteristic point on the scale mark marked by the corresponding scale number, that is, when the pointer points to the key point of the scale number, the instrument reading is the scale number corresponding to the key point of the scale number pointed by the pointer.
When the front view dial, the pointer rotating shaft central point, the pointer pointing line, the scale number and the corresponding scale number key point are obtained according to the pointer instrument image to be detected, referring to fig. 4, the dial, the pointer rotating shaft central point, the scale number and the corresponding scale number key point in the pointer instrument image to be detected can be detected first. The dial plate in the input pointer instrument image to be measured is firstly positioned, and the scale number key point taking the scale number as the center in the dial plate is extracted, and correspondingly, the scale number key point can be the key point on the scale mark marked by the scale number. In this step, the rotation axis center point of the pointer and the key point on the pointer trunk need to be extracted from the input pointer instrument image to be measured, for example, two end points at two ends of the pointer trunk and the key point such as the middle point in the length direction of the pointer trunk can be extracted. And then carrying out ellipse fitting on the scale number key points to obtain a fitting ellipse. The scale digital key points can be subjected to ellipse fitting through a least square method, and distortion correction is performed on a dial plate in an image of the pointer instrument to be detected. And then, obtaining the front view dial plate according to the central point of the pointer rotating shaft, the fitting ellipse and the key point of the scale number. The calibration of the pointer instrument image to be measured can be realized by using the scale number key points on the current instrument image and the center point of the pointer rotating shaft. The method does not need to use a same type of front view instrument as a template and then carry out correction in a characteristic point matching mode like the prior art, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved.
When the front view dial is obtained according to the central point of the pointer rotating shaft, the fitting ellipse and the scale number key point, referring to fig. 4, the distortion of the dial in the image of the pointer instrument to be detected can be corrected first, and then the rotation correction is carried out. When distortion correction is carried out, a perspective transformation correction algorithm can be adopted, and all points on the dial plate in the pointer instrument image to be measured are projected onto the assumed dial plate plane according to the pointer rotating shaft central point and the fitting ellipse. Firstly, a plane where a dial is located is assumed as an assumed surface plane, a rectangular coordinate system is established on the assumed dial plane by taking a central point of a pointer rotating shaft as an original point, a circle is drawn by taking the central point of the pointer rotating shaft as a circle center and a preset radius r, and the circle and a horizontal and vertical coordinate axis in the rectangular coordinate system are intersected at four intersection points. And solving a perspective transformation matrix H by utilizing the corresponding relation between the four vertexes of the long axis and the short axis of the fitting ellipse obtained in the step and the four intersection points on the horizontal and vertical coordinates, and projecting all points on the dial plate in the image of the pointer instrument to be detected onto the plane of the assumed dial plate by utilizing the perspective transformation matrix H to realize distortion correction of the dial plate in the image of the pointer instrument to be detected.
When the rotation correction is performed, with reference to fig. 4, the dial projected on the assumed dial plane can be processed by using the symmetry of the scale number key points and the rotation correction algorithm, so as to obtain the front view dial. Due to the mode of projecting all the points on the dial plate in the image of the instrument to be measured onto the assumed dial plate plane, the projected dial plate plane is parallel to the assumed dial plate plane. There may be instances where the projected meter dial will rotate, causing the projected dial plane to have a rotational angle difference from the assumed dial plane. At this time, the symmetry of the scale number key points can be utilized to connect the minimum scale (usually zero scale) and the maximum scale (usually full scale) on the scale number key points to form a line of the minimum scale key points and the maximum scale key points, the rotation angle of the line relative to the horizontal direction is calculated, and then the rotation correction of the instrument is realized according to the rotation angle to obtain the front view dial plate of the instrument. The key information on the current instrument image is conveniently utilized to quickly correct the pointer instrument image to be detected to obtain the front view dial plate.
In addition, in the process of obtaining the pointer direction line according to the image of the pointer instrument to be measured, referring to fig. 4, the direction of the pointer may be judged first, then the key points of the pointer trunk are extracted, and the pointer direction line is determined according to the direction of the pointer. For example, referring to fig. 2, two end points (e.g., end point M and end point N in fig. 2) on the pointer stem on the front view dial can be detected first. Then, the middle point of the pointer stem (e.g., point C in fig. 2) is obtained from the two end points. Then, the pointing direction of the pointer and the pointer pointing line are determined according to the pointer rotation axis center point (such as the point O in fig. 2) and the middle point of the pointer trunk. The pointer direction and the pointer direction line can be quickly obtained by using the pointer key point information on the current instrument image.
When the pointing direction of the pointer and the pointer pointing line are determined according to the central point of the pointer rotating shaft and the central point of the pointer trunk, referring to fig. 2, the pointing direction of the pointer can be determined by establishing a rectangular coordinate system and comparing the coordinate difference between the central point of the pointer rotating shaft and the central point of the pointer trunk, and then the pointer pointing line is determined according to the pointer pointing direction and the end point on the pointer trunk. Specifically, a rectangular coordinate system may be first established on the face plate. And then, detecting the coordinates of the central point of the pointer rotating shaft and the central point of the pointer trunk on the rectangular coordinate system. And then, determining the direction of the pointer according to the coordinates of the central point of the rotating shaft of the pointer and the central point of the main trunk of the pointer. And then, a connecting line of the central point of the pointer rotating shaft and the front end point of the pointer trunk is used as a pointer pointing line (the pointing line pointing to the N point from the O point represents the pointer pointing line). The point of the pointer can be quickly and accurately determined by using the horizontal and vertical coordinate difference of the central point of the pointer rotating shaft and the central point of the pointer trunk under the rectangular coordinate system.
When the rectangular coordinate system is established on the front view dial plate, various modes can be adopted. For example, as shown in fig. 2, an x-y rectangular coordinate system may be established on the face plate of the face plate, with the central point (O point) of the pointer rotation axis as the origin, and the parallel direction of the connecting lines of the key points (points a and B) of the scale numbers corresponding to the minimum scale number and the maximum scale number on the face plate of the face plate as the x axis, such that point D in fig. 2 is located on the y axis and is at the same time concentric with the key point A, B of the scale numbers. The direction of the pointer can be determined conveniently and quickly according to the horizontal and vertical coordinate difference between the central point (point O) of the rotating shaft of the pointer and the central point (point C) of the main body of the pointer, and the direction judgment mechanism table shown below can be specifically adopted to judge that the pointer points to the fourth quadrant in the rectangular coordinate system. It is to be explained that the coordinate of the point C therein is (C) x ，c y ) The coordinate of the point O is (O) x ，o y ) The angle θ in table 1 is the size of the angle formed by clockwise rotation of OD with OD as the starting axis in fig. 2.
TABLE 1 Direction decision mechanism Table
Of course, fig. 2 also shows another method for establishing a rectangular coordinate system on the face plate, which may be to arbitrarily select a point on the face plate as an origin, and similarly, establish a U-V rectangular coordinate system as shown in fig. 2 by using a parallel direction of connecting lines of scale number key points (points a and B) corresponding to the minimum scale number and the maximum scale number on the face plate as a U axis on the rectangular coordinate system. The specific judgment mode is the same as the above mode, and is not described herein again.
Next, referring to fig. 1, two scale number key points which are arranged on the front view dial plate at both sides of the pointer direction line and are closest to the front end of the pointer direction line are detected. The scale numbers corresponding to the key points of the two scale numbers are respectively as follows: a first scale number, and a second scale number greater than the first scale number. Specifically, the two scale number key points which are respectively arranged at the two sides of the pointer pointing line and are closest to the front end of the pointer pointing line can be obtained by obtaining the rectangular frame of the two scale numbers which are closest to the left and the right sides of the pointer pointing line, dividing the rectangular frame and putting the rectangular frame into a LeNet-5 digital identification model for identification. As shown in fig. 3, the identified key points of the scale numbers listed ON both sides of the pointer direction line ON and closest to the pointer direction line ON are respectively a point b and a point c, the point a pointed by the pointer direction line ON is located ON the scale circle of the dial, i.e., the points a, b and c are a common circle, and the center of the circle is the pointer rotation shaft center point O.
Next, referring to fig. 1, the current meter reading of the meter image of the pointer to be measured is calculated mainly by using a local angle method. Specifically, as shown in fig. 1 and 3, the included angle formed by the central point of the rotating shaft of the pointer and the connecting line between the key points of the two scale numbers is detected. Starting from the center point of the pointer rotating shaft, the nearest scale number key points on the left side and the right side of the pointer pointing line are respectively connected, an included angle is formed at the center point of the pointer rotating shaft, and the included angle is represented by ≈ bOc in fig. 3 and represents the arc angle span size between the nearest scale number key points on the left side and the right side of the pointer pointing line.
Then, with reference to fig. 1 and fig. 3, two sub-angles formed by dividing the included angle by the pointer-pointing line are detected. Starting from the center point of the pointer rotating shaft, the nearest scale number key points on the left side and the right side of the pointer pointing line are respectively connected, and the included angle formed at the center point of the pointer rotating shaft can be divided into two sub-included angles by the pointer pointing line. As shown in fig. 3, a pointer pointing line ON divides ≤ bOc into ≤ bOa and ≤ aOc, and the ≤ bOa and ≤ aOc are used as two sub-angles of ≤ bOc, which are respectively arranged at two sides of the pointer pointing line to represent the arc angle span between the pointer pointing line and the two scale number key points.
Then, as shown in fig. 1 and 3, the current meter reading of the pointer meter is calculated according to the included angle, the two sub-included angles and the two scale numbers. According to the included angle, the two sub-included angles and the two scale numbers, calculating the current instrument reading number of the pointer instrument comprises the following steps: calculating the current instrument reading value of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers by adopting the following formula:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Represents the first scale number, for example, the scale number corresponding to the scale number key point b in fig. 3 is V 1 ；V 2 Represents a second scale number, for example, the scale number corresponding to the scale number key point c in fig. 3 is V 2 (ii) a α represents an included angle, such as α ═ bOc in fig. 3; alpha is alpha 1 Indicating that of the two sub-angles, the scale number key point corresponding to the first scale number is located at the sub-angle on the same side of the pointer-pointing line, e.g. α in fig. 3 1 Equal to bOa. The current meter reading of the pointer meter is calculated and obtained quickly and accurately through a local angle method.
It will be appreciated that the above only shows one way of calculating the meter reading, but that other ways are possible, for example the following formula may be used to calculate the current meter reading value of the pointer meter:
value＝V 2 -(V 2 -V 1 )×α 2 ÷α
wherein alpha is 2 Indicating that of the two sub-angles, the scale number key point corresponding to the second scale number is located at the sub-angle on the same side of the pointer-pointing line, e.g. α in fig. 3 2 ＝∠cOa。
In the various embodiments shown above, the image of the pointer instrument to be measured is processed to obtain a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale number key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front-view dial plate, and calculating to obtain the current instrument reading of the pointer instrument through a local angle method according to an included angle formed by the central point of a pointer rotating shaft and a connecting line between the two scale number key points, two sub-included angles formed by the included angle divided by the pointer direction line and scale numbers corresponding to the two scale number key points. The method does not need to take an orthophoria instrument of the same type as the prior art as a template, then carries out correction in a characteristic point matching mode, and carries out the number indicating calculation by manually inputting the measuring range of the instrument. The method disclosed by the application can acquire the key information for calculating the instrument readings from the current instrument image without prior information, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved. In addition, the key points of the scale numbers and the scale numbers on the left side and the right side of the pointer direction line are identified, and then the meter readings are calculated by using a local angle method, so that the method and the device are suitable for meters with uniform scales and non-uniform scales.
In addition, an embodiment of the present invention further provides a pointer and meter reading system, and referring to fig. 5, the pointer and meter reading system includes: the device comprises an image acquisition module, a detection and dial plate correction module, a scale number key point identification module, an included angle identification module and a number indication calculation module. The image acquisition module is used for acquiring an image of the pointer instrument to be detected. The detection and dial plate correction module is used for obtaining a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected. The scale number key point identification module is used for detecting two scale number key points which are arranged on the front-view dial plate and are respectively arranged on two sides of the pointer direction line and are closest to the front end of the pointer direction line. The included angle identification module is used for detecting included angles formed by the central point of the pointer rotating shaft and a connecting line between the central point and the two scale number key points respectively; and the device is also used for detecting two sub-included angles formed by dividing the included angle by the pointer pointing line. And the number-indicating calculation module is used for calculating the current instrument number of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
In the scheme, the image of the pointer instrument to be detected is processed to obtain a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points; and then detecting two scale digital key points which are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front view dial plate, and calculating the current instrument reading number of the pointer instrument by a local angle method according to an included angle formed by connecting the central point of the pointer rotating shaft with the connecting line between the two scale digital key points, two sub-included angles formed by dividing the included angle by the pointer direction line and scale numbers corresponding to the two scale digital key points. The method does not need to take an orthophoria instrument of the same type as the prior art as a template, then carries out correction in a characteristic point matching mode, and carries out the number indicating calculation by manually inputting the measuring range of the instrument. The method disclosed by the application can acquire the key information for calculating the instrument readings from the current instrument image without prior information, so that a large amount of accurate labels are not needed when a data set is manufactured, and a large amount of labor and time can be saved. In addition, the key points of the scale numbers and the scale numbers on the left side and the right side of the pointer direction line are identified, and then the meter readings are calculated by using a local angle method, so that the method and the device are suitable for meters with uniform scales and non-uniform scales.
Referring to fig. 5 and 6, the detection and dial plate correction module may include: detection module and dial plate correction module. The detection module is used for positioning the pointer instrument image to be detected based on the improved YOLOv4 model, and detecting the central point of the pointer rotating shaft, the end point on the pointer trunk, the scale number and the corresponding key point of the scale number. Fig. 8 shows a network structure diagram of an improved YOLOv4 model. The improved YOLOv4 model consists of a CSPDarknet backbone Network (Cross Stage Partial Darknet), an SPP (spatial Pyramid pooling), a PANET (Path Aggregation Network) multi-scale feature extraction module and a YOLO Head output layer. Contact in fig. 8 represents concatenation, Upsampling in fig. 8 represents Upsampling, Downsampling in fig. 8 represents Downsampling, Max posing in fig. 8 represents Max pooling, Resblock _ body in fig. 8 represents residual unit, and Conv in fig. 8 represents convolution. Referring to fig. 7, the standard convolution in the CSPDarknet backbone network may be replaced by a depth separable convolution. And the dial plate correction module is used for obtaining a front view dial plate according to the pointer instrument image to be detected. The parameter calculation amount can be reduced to a greater extent, and the target detection speed is improved; meanwhile, a 104 x 104 characteristic layer can be added in the original YOLOv4 prediction network structure to obtain richer characteristic information and improve the capability of the network for detecting small targets.
In forming the detection module described above, a data set can be created. Specifically, firstly, a certain number of pictures containing the pointer instrument are adopted, rectangular frame labeling and key point labeling are respectively carried out on the dial plate, the scale numbers and the corresponding scale number key points of each image, and key points (such as the end points of the pointer and the central point of the pointer rotating shaft) on the pointer are labeled. It is also possible to make a print number set that contains mainly fractional and integer numbers in order to detect fractional and integer numbers in the scale numbers. Thereafter, model training may be performed. Inputting the digital set labeled in the previous step into an improved YOLOv4 model for training, performing data enhancement processing during the training process, and obtaining a detection model as a detection module after the training is completed. In addition, the number set manufactured in the previous step can be input into a LeNet-5 model for training to obtain a number recognition model, so that the nearest scale numbers and corresponding scale number key points which are arranged on the left side and the right side of the pointer direction line in a subsequent step can be recognized.
As shown in fig. 6, a direction determination module may be further disposed in the meter reading system to determine the direction of the pointer according to the determination method described in the meter reading method section. Of course, a number identification module can be arranged in the instrument number reading system to identify the nearest scale numbers and corresponding scale number key points which are respectively arranged at the left side and the right side of the pointer pointing line.
In addition, as described in the above method section, the scale numbers corresponding to the two scale number key points are: a first scale number, and a second scale number greater than the first scale number. At this time, the number calculation module can calculate the current meter number value of the pointer meter according to the included angle, the two sub-included angles and the two scale numbers by adopting the following formula:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Representing a first scale number; v 2 To representA second scale number; alpha represents an included angle; alpha is alpha 1 And the scale number key point corresponding to the first scale number in the two sub-included angles is positioned in the sub-included angle on the same side of the pointer pointing line. The current meter reading of the pointer meter is calculated and obtained quickly and accurately through a local angle method.
It should be noted that the image acquisition module, the detection and dial plate correction module, the scale number key point identification module, the included angle identification module and the number indication calculation module are all software and hardware assemblies capable of realizing corresponding functional methods. That is, each functional module includes not only corresponding hardware such as a circuit board and a chip, but also software codes stored and run on the hardware. When the function module realizes the corresponding function, all the functions described in the corresponding steps in the pointer and meter reading method can be realized as described in the pointer and meter reading method.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A pointer instrument reading method is characterized by comprising the following steps:
acquiring an image of a pointer instrument to be detected;
obtaining a front view dial, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected;
detecting two scale digital key points which are respectively arranged on two sides of the pointer direction line and are closest to the front end of the pointer direction line on the front view dial plate;
detecting included angles formed by the central point of the pointer rotating shaft and a connecting line between the central point and the two scale number key points respectively;
detecting two sub-included angles formed by dividing the included angle by the pointer pointing line;
and calculating the current instrument reading of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
2. The method for reading the number of the pointer instrument as claimed in claim 1, wherein the obtaining of the front view dial, the center point of the pointer rotation shaft, the pointer direction line, the scale number and the corresponding key point of the scale number according to the image of the pointer instrument to be measured comprises:
detecting a dial plate, a pointer rotating shaft central point, scale numbers and corresponding scale number key points in the pointer instrument image to be detected;
carrying out ellipse fitting on the scale number key points to obtain a fitting ellipse;
and obtaining the front view dial according to the central point of the pointer rotating shaft, the fitting ellipse and the scale number key point.
3. The method for reading the indication number of the pointer instrument as claimed in claim 2, wherein the obtaining the front view dial according to the central point of the rotating shaft of the pointer, the fitting ellipse and the key point of the scale number comprises:
projecting all points on the dial plate in the pointer instrument image to be detected onto a hypothetical dial plate plane by adopting a perspective transformation correction algorithm according to the central point of the pointer rotating shaft and the fitting ellipse;
and processing the dial projected on the assumed dial plane by using the symmetry of the scale number key points and a rotation correction algorithm to obtain the front view dial.
4. The method for reading the number of the pointer instrument as claimed in claim 2 or 3, wherein the obtaining of the front view dial, the center point of the pointer rotation shaft, the pointer direction line, the scale number and the corresponding key point of the scale number according to the image of the pointer instrument to be measured further comprises:
detecting two end points on the pointer trunk on the front view dial plate;
obtaining a middle point of the pointer trunk according to the two end points;
and determining the direction of the pointer and the pointer direction line according to the central point of the pointer rotating shaft and the middle point of the pointer trunk.
5. The pointer instrument indication reading method of claim 4, wherein said determining the pointer direction and the pointer direction line based on the pointer shaft center point and the pointer stem center point comprises:
establishing a rectangular coordinate system on the front view dial plate;
detecting coordinates of the central point of the pointer rotating shaft and the midpoint of the pointer trunk on the rectangular coordinate system;
determining the direction of the pointer according to the coordinates of the central point of the pointer rotating shaft and the middle point of the pointer trunk;
and taking the connection line of the central point of the pointer rotating shaft and the front end point of the pointer trunk as the pointer pointing line.
6. A pointer instrument reading method according to claim 5, wherein said establishing a rectangular coordinate system on said face plate comprises:
and taking the central point of the pointer rotating shaft as an original point, taking the parallel direction of the scale number key point connecting lines respectively corresponding to the minimum scale number and the maximum scale number on the face of.
7. The reading method for the number of the indicating instrument of any one of claims 1 to 6, wherein the corresponding scale numbers of the two scale number key points are respectively: a first scale number, and a second scale number greater than the first scale number;
calculating the current instrument reading of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers comprises:
and calculating the current instrument reading value of the pointer instrument by adopting the following formula according to the included angle, the two sub-included angles and the two scale numbers:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Representing the first scale number;
V 2 representing the second scale number;
α represents the angle;
α 1 and indicating the sub-included angle of which the scale number key point corresponding to the first scale number is positioned on the same side of the pointer pointing line in the two sub-included angles.
8. A pointer instrument reading system, comprising:
the image acquisition module is used for acquiring an image of the pointer instrument to be detected;
the detection and dial plate correction module is used for obtaining a front view dial plate, a pointer rotating shaft central point, a pointer pointing line, scale numbers and corresponding scale number key points according to the pointer instrument image to be detected;
the scale number key point identification module is used for detecting two scale number key points which are arranged on the front-view dial plate and are respectively arranged at two sides of the pointer direction line and are closest to the front end of the pointer direction line;
the included angle identification module is used for detecting included angles formed by the central point of the pointer rotating shaft and a connecting line between the central point and the two scale number key points respectively; the device is also used for detecting two sub-included angles formed by the division of the included angle by the pointer pointing line;
and the number calculation module is used for calculating the current instrument number of the pointer instrument according to the included angle, the two sub-included angles and the two scale numbers.
9. The pointer instrument reading system of claim 8 wherein said detection and dial correction module comprises:
the detection module is used for positioning the pointer instrument image to be detected based on the improved YOLOv4 model, and detecting the central point of the pointer rotating shaft, the end point on the pointer trunk, the scale number and the corresponding scale number key point; the improved YOLOv4 model consists of a CSPDarknet backbone network, an SPP space pyramid pooling module, a PANET multi-scale feature extraction module and a YOLO Head output layer; the standard convolution in the CSPDarknet backbone network is replaced by a depth separable convolution;
and the dial plate correction module is used for obtaining a front view dial plate according to the pointer instrument image to be detected.
10. The pointer instrument reading system of claim 8, wherein the two scale number key points correspond to scale numbers respectively: a first scale number, and a second scale number greater than the first scale number;
the number calculation module calculates the current meter number value of the pointer meter by adopting the following formula according to the included angle, the two sub-included angles and the two scale numbers:
value＝V 1 +(V 2 -V 1 )×α 1 ÷α
wherein, V 1 Representing the first scale number;
V 2 representing the second scale number;
α represents the angle;
α 1 and indicating the sub-included angle of which the scale number key point corresponding to the first scale number is positioned on the same side of the pointer pointing line in the two sub-included angles.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210495907.4A CN114882487A (en) | 2022-04-27 | 2022-04-27 | Pointer instrument reading method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210495907.4A CN114882487A (en) | 2022-04-27 | 2022-04-27 | Pointer instrument reading method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114882487A true CN114882487A (en) | 2022-08-09 |
Family
ID=82673677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210495907.4A Pending CN114882487A (en) | 2022-04-27 | 2022-04-27 | Pointer instrument reading method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114882487A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115797934A (en) * | 2022-12-01 | 2023-03-14 | 北京百度网讯科技有限公司 | Instrument number indicating method and device, electronic equipment and storage medium |
-
2022
- 2022-04-27 CN CN202210495907.4A patent/CN114882487A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115797934A (en) * | 2022-12-01 | 2023-03-14 | 北京百度网讯科技有限公司 | Instrument number indicating method and device, electronic equipment and storage medium |
| CN115797934B (en) * | 2022-12-01 | 2023-12-01 | 北京百度网讯科技有限公司 | Meter registration method, apparatus, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110659636A (en) | Pointer instrument reading identification method based on deep learning | |
| Zheng et al. | A robust and automatic recognition system of analog instruments in power system by using computer vision | |
| CN105701766B (en) | Image matching method and device | |
| CN106599897B (en) | Readings of pointer type meters recognition methods and device based on machine vision | |
| CN111414934A (en) | Pointer type meter reading automatic identification method based on fast R-CNN and U-Net | |
| CN112257676B (en) | Pointer type instrument reading method and system and inspection robot | |
| CN111950330A (en) | Pointer instrument indicating number detection method based on target detection | |
| CN101673397B (en) | Digital camera nonlinear calibration method based on LCDs | |
| CN104657711B (en) | A kind of readings of pointer type meters automatic identifying method of robust | |
| CN110675376A (en) | PCB defect detection method based on template matching | |
| CN109558871B (en) | Pointer instrument reading identification method and device | |
| CN111368906B (en) | Pointer type oil level meter reading identification method based on deep learning | |
| CN109284718B (en) | Inspection robot-oriented variable-view-angle multi-instrument simultaneous identification method | |
| CN110852954B (en) | Image inclination correction method and system for pointer instrument | |
| Yang et al. | An image-based intelligent system for pointer instrument reading | |
| CN110245654A (en) | Total indicator reading recognition methods and device, computer installation and readable storage medium storing program for executing | |
| CN111401357B (en) | Pointer type instrument reading method based on text detection | |
| CN110211178A (en) | A kind of pointer instrument recognition methods calculated using projection | |
| CN114882487A (en) | Pointer instrument reading method and system | |
| Zhuo et al. | Machine vision detection of pointer features in images of analog meter displays | |
| CN111091121B (en) | Ellipse dial plate detection and correction method based on image processing | |
| CN115588196A (en) | Pointer type instrument reading method and device based on machine vision | |
| CN115512343A (en) | Method for correcting and recognizing reading of circular pointer instrument | |
| CN112464986B (en) | Reading method and device for pointer type disc instrument | |
| CN113050022B (en) | Image positioning method and device based on rotary antenna and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |