CN115393838A - Pointer instrument reading identification method and device, electronic equipment and storage medium - Google Patents
Pointer instrument reading identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115393838A CN115393838A CN202111088339.8A CN202111088339A CN115393838A CN 115393838 A CN115393838 A CN 115393838A CN 202111088339 A CN202111088339 A CN 202111088339A CN 115393838 A CN115393838 A CN 115393838A
- Authority
- CN
- China
- Prior art keywords
- dial
- image
- graph
- template
- pointer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a pointer instrument reading identification method, a pointer instrument reading identification device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an input image of a dial plate of the pointer instrument; inputting an input image into a pre-trained target detection model, and detecting an instrument dial area in the input image to obtain a dial subgraph; and rotationally matching the dial sub-graph with the template dial graph, calculating matching confidence coefficients under a plurality of rotation angles, sequencing according to a sequence from large to small, calculating the image feature similarity of the dial sub-graph and the template dial graph under the angle for each angle in the rotation angles corresponding to the first n matching confidence coefficients to obtain n feature similarity results, calculating the maximum value of the results, determining the pointer identification angle of the pointer instrument, and converting the pointer identification angle into a reading value according to the scale information of the instrument. Therefore, the method can support reading identification of the pointer instrument with multiple scales, multiple dials and multiple pointers, has small calculation amount and is suitable for multiple scenes.
Description
Technical Field
The invention relates to the technical field of pointer instrument identification, in particular to a pointer instrument reading identification method and device, electronic equipment and a storage medium.
Background
The pointer instrument is used as a traditional measuring instrument, is widely applied to the fields of electric power systems, industrial production, scientific experiments and the like, and can indicate the running state in real time. Most of the existing pointer instruments basically read meters manually, so that the workload is large, the efficiency is low, errors are easy to occur, and the pointer instruments are not suitable for manually reading meters in high-temperature, high-voltage and high-radiation environments such as a transformer substation.
In the prior art, a circle in a detection image can be used as a dial area through Hough transformation, a pointer straight-line segment in the image is further detected, and then a pointer angle is calculated by combining the dial circle area and the pointer area to obtain a meter reading; and the dial plate and pointer readings under all scales of a certain type can be marked manually, a deep learning target detection model is trained, and the reading identification is directly carried out through the target detection model.
However, the hough transform-based method needs to adjust a plurality of manual threshold values, is not stable enough, and requires that the straight-line segment of the pointer is long enough, otherwise, the detection failure is easily caused or the detection precision is not high, the practicability is not strong, and the method for performing deep learning by manual labeling needs to manually provide a large amount of data, so that the labor intensity of workers is high, the scene migration is difficult, and various types of instruments cannot be recognized simultaneously.
Disclosure of Invention
The invention provides a reading identification method and device for a pointer instrument, electronic equipment and a storage medium, which can be suitable for reading identification of the pointer instrument with multiple scales, multiple dials and multiple pointers, have strong applicability and are suitable for various scenes.
In a first aspect, the present invention provides a pointer instrument reading identification method, including: acquiring an input image of a dial plate of the pointer instrument; inputting the input image into a pre-trained target detection model, and detecting an instrument dial area in the input image to obtain a dial sub-graph; the target detection model is a deep learning model constructed based on a neural network; carrying out rotation matching on the dial sub-graph and the template dial graph, and calculating matching confidence coefficients under a plurality of rotation angles; sequencing the matching confidence degrees under the multiple rotation angles according to a sequence from large to small, and calculating the image feature similarity of the dial sub-graph and the template dial sub-graph under each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results; n is a positive integer greater than 1; and determining a pointer identification angle of the pointer instrument according to the maximum value in the n feature similarity results, and converting the pointer identification angle into a reading value according to instrument scale information.
Optionally, the method further includes: acquiring a training data set, wherein the training data set comprises a dial plate image, a dial plate area, a corresponding dial plate type and the number of pointers in the dial plate; training a target detection model according to the training data set; correspondingly, inputting the input image into a pre-trained target detection model, detecting an instrument dial area in the input image, and obtaining a dial subgraph, wherein the steps of: inputting the input image into a target detection model obtained by training the training data set, detecting an instrument dial area in the input image to obtain a dial sub-graph, and determining the type of the dial and the number of pointers; and determining a template dial chart according to the dial type and the number of the pointers.
Optionally, the method further includes: normalizing the size of the meter dial area to the size of a template dial map, and/or normalizing the size of a pointer in the meter dial area to the size of a pointer in the template dial map; the template dial image is an image with preset size.
Optionally, the rotating and matching of the dial sub-graph and the template dial sub-graph includes: extracting the gray level image of the dial sub-graph, and acquiring a gray level template of the template dial sub-graph; and rotating the gray level image of the dial sub-graph by 0-360 degrees to match with the gray level template, or rotating the gray level template by 0-360 degrees to match with the gray level image of the dial sub-graph.
Optionally, calculating the matching confidence degrees at a plurality of rotation angles includes: acquiring a first local feature of the dial sub-graph and a second local feature of the template dial sub-graph; the first local feature and the second local feature are both image features of the region where the pointer is located; calculating a confidence of the matching of the second local feature to the first local feature at a plurality of rotation angles.
Optionally, the image feature is a feature of a color image; calculating the image feature similarity of the dial sub-graph and the template dial sub-graph at each angle in the rotation angles corresponding to the first n matching confidences, wherein the image feature similarity comprises the following steps: acquiring the characteristics of the template dial images corresponding to the first n matching confidence degrees and the color images of the dial sub-images, which are extracted by an image characteristic extractor, wherein the image characteristic extractor is an image characteristic extractor trained on the basis of a deep learning model; and calculating the similarity of the characteristics of the color image of the template dial chart corresponding to the first n matching confidences and the characteristics of the color image of the dial sub-chart.
Optionally, the meter scale information is a prestored comparison table corresponding to a reading value in each angle interval.
In a second aspect, the present invention provides a pointer instrument reading identification apparatus, the apparatus comprising: the acquisition module is used for acquiring an input image of the dial plate of the pointer instrument; the detection module is used for inputting the input image into a pre-trained target detection model, detecting an instrument dial area in the input image and obtaining a dial sub-graph; the target detection model is a deep learning model constructed based on a neural network; the calculation module is used for carrying out rotation matching on the dial sub-graph and the template dial graph and calculating matching confidence coefficients under a plurality of rotation angles; the processing module is used for sequencing the matching confidence degrees under the plurality of rotation angles from large to small, and calculating the image feature similarity of the dial sub-graph and the template dial sub-graph under each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results; n is a positive integer greater than 1; and the determining module is used for determining the pointer identification angle of the pointer instrument according to the maximum value in the n feature similarity results and converting the pointer identification angle into a reading value according to instrument scale information.
In a third aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the pointer meter reading identification method of any of the first aspects.
In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions for implementing the pointer meter reading identification method according to any one of the first aspect when the computer-executable instructions are executed by a processor.
In summary, the present invention provides a method, an apparatus, an electronic device and a storage medium for identifying reading of a pointer instrument, wherein the method can obtain an input image of a dial of the pointer instrument; further, inputting the input image into a pre-trained target detection model, and detecting an instrument dial area in the input image to obtain a dial subgraph; and then, carrying out rotation matching on the dial sub-graph and the template dial graph, calculating matching confidence degrees under a plurality of rotation angles, sequencing the rotation angles from large to small, calculating the image feature similarity of the dial sub-graph and the template dial graph under the angle for each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results, determining the pointer identification angle of the pointer instrument according to the maximum value in the n feature similarity results, and converting the pointer identification angle into a reading value according to the instrument scale information. Therefore, the method is suitable for reading identification of multi-scale, multi-dial and multi-pointer type instruments, small in calculation amount, suitable for various scenes and high in accuracy.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a schematic view of an application scenario of a pointer instrument reading identification method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a pointer instrument reading identification method according to an embodiment of the present invention;
fig. 3 is a flowchart of a reading identification method for a pointer instrument according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a pointer instrument reading identification apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
With the above figures, certain embodiments of the invention have been illustrated and described in more detail below. The drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
In order to facilitate clear description of technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish the same items or similar items having substantially the same functions and actions. For example, the first device and the second device are only used for distinguishing different devices, and the order of the devices is not limited. Those skilled in the art will appreciate that the terms "first," "second," and the like do not denote any order or importance, but rather the terms "first," "second," and the like do not denote any order or importance.
It is to be understood that the terms "exemplary" or "such as" are used herein to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present relevant concepts in a concrete fashion.
In the present invention, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a alone, A and B together, and B alone, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
Embodiments of the present invention will be described below with reference to the accompanying drawings. Fig. 1 is a schematic view of an application scenario of a pointer instrument reading identification method provided in an embodiment of the present invention, and the pointer instrument reading identification method provided in the present invention may be applied to the application scenario shown in fig. 1. The application scenario includes: pointer instrument 101 and pointer instrument 102, camera 103, server 104, terminal device 105 and user 106. In this scenario, the camera 103 may capture dial images of the pointer instrument 101 and the pointer instrument 102, and send the captured dial images to the server 104 for processing, further, the server 104 may receive the dial images and perform meter dial reading recognition, further, the server 104 may send the recognized results to the terminal device 105 having a display screen, and the terminal device 105 receives the recognized results and displays the recognized results on the screen for the user 106 to view.
It can be understood that the camera 103 may first shoot the dial plate image of the pointer instrument 101 according to a preset period, and then shoot the dial plate image of the pointer instrument 102 for processing in sequence, or may manually control the camera 103 to first shoot the dial plate image of the pointer instrument 102 and then shoot the dial plate image of the pointer instrument 101.
It should be noted that the terminal device 105 may be a large screen (or called an intelligent screen), a mobile phone, a tablet computer, a smart watch, a smart bracelet, smart glasses, or other terminal devices with a display screen, and the embodiment of the present invention is not limited thereto.
In the prior art, a circle in an input image can be detected through Hough transform as a dial area, a pointer straight-line segment in the image is further detected, and then a pointer angle is calculated by combining the dial circle area and the pointer area to obtain a meter reading; the dial plate and the pointer reading under all scales of a certain type can be marked manually to train a deep learning target detection model, and furthermore, the shot picture is input into the target detection model to directly perform reading recognition.
However, the hough transform-based method needs to adjust a plurality of manual threshold values, is not stable enough, and requires that the straight-line segment of the pointer is long enough, otherwise, the detection failure is easily caused or the detection precision is not high, the practicability is not strong, and the method for performing deep learning by manual labeling needs to manually provide a large amount of data, so that the labor intensity of workers is high, the scene migration is difficult, and various types of instruments cannot be recognized simultaneously.
Therefore, the invention provides a reading identification method of a pointer instrument, which can input the acquired input image of a dial plate of the pointer instrument into a pre-trained target detection model, and further detect the area of the dial plate of the instrument in the input image to obtain a dial plate subgraph; and then, carrying out rotation matching on the dial sub-graph and the template dial graph, calculating matching confidence degrees under a plurality of rotation angles, sequencing the rotation angles from large to small, calculating the image feature similarity of the dial sub-graph and the template dial graph at each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results, determining a pointer identification angle of the pointer instrument according to the maximum value in the n feature similarity results, and further converting the pointer identification angle into a reading value according to the instrument scale information to facilitate the user to check.
Exemplarily, fig. 2 is a schematic flow chart of a pointer instrument reading identification method according to an embodiment of the present invention, and as shown in fig. 2, the method according to the embodiment of the present invention includes:
s201, obtaining an input image of a dial of the pointer instrument.
In the embodiment of the present invention, the pointer type meter may refer to an instrument that displays a measured value by movement of a pointer, and may be used to indicate the measured value, which is various, for example, a bourdon tube pressure gauge, a bimetal thermometer, a thermometer bulb, a moving coil type indicator, and the like.
For example, in the application scenario of fig. 1, the server 104 may acquire input images of the dials of the pointer instrument 101 and the pointer instrument 102 captured by the camera 103.
S202, inputting the input image into a pre-trained target detection model, and detecting an instrument dial area in the input image to obtain a dial sub-graph; the target detection model is a deep learning model constructed based on a neural network.
In the embodiment of the invention, the neural network is an important machine learning technology, the network structure of the neural network comprises an input layer, a hidden layer and an output layer, and the number of layers of the hidden layer can be set. The training process of the neural network mainly utilizes the principle of back propagation to carry out network gradient descent optimization so as to find the best model parameters, and the neural network can be used for image recognition, voice recognition, text recognition and the like.
Deep learning models may refer to architectures based on a deep learning algorithm that may be used to iteratively train data. For example, yolov5 model, yolov4 model, SSD model, faster-RCNN model, and the like.
For example, in the application scenario of fig. 1, the server 104 may input the dial image of the pointer instrument 101 captured by the camera 103 into a previously trained yolov5 model, and further detect an instrument dial area in the dial image of the pointer instrument 101 to obtain a dial subgraph.
It can be understood that the target monitoring model of the invention can identify instrument panels with multiple scales, multiple dials and multiple pointers, and has wide application range.
S203, carrying out rotation matching on the dial sub-graph and the template dial sub-graph, and calculating matching confidence degrees under a plurality of rotation angles.
In the embodiment of the present invention, the template dial map may refer to a training data set trained in advance by the target detection model, and the training data set may include various dial types and the number of pointers in the dial. The matching confidence may refer to a value for measuring the correctness of the matching result.
For example, the server may perform rotation matching on the dial sub-image detected by the target detection model and the template dial image, and calculate matching confidence degrees at a plurality of rotation angles, for example, the dial sub-image may be sequentially rotated clockwise by 1 degree to obtain 360 images, which are respectively compared with the images in the template dial image one by one, and further calculate matching confidence degrees of the 360 images and the images in the template dial image, where the matching confidence degree may be a matching confidence degree of a pointer region or a matching confidence degree of the entire region.
It should be noted that, in the embodiment of the present invention, the rotation dial sub-diagram and the template dial sub-diagram are not specifically limited, the rotation dial sub-diagram may be matched with the template dial sub-diagram, and the rotation template dial sub-diagram may also be matched with the template dial sub-diagram, and it should be further understood that the present invention does not limit specific rotation degrees and rotation modes, and may rotate clockwise every few degrees, and may also rotate counterclockwise according to a certain rule, for example, rotate 1 degree counterclockwise for the first time, rotate 2 degrees counterclockwise for the second time, rotate 3 degrees counterclockwise for the third time, and rotate N degrees counterclockwise up to the nth time, and the like.
S204, sequencing the matching confidence degrees under the multiple rotation angles according to a sequence from large to small, and calculating the image feature similarity of the dial sub-graph and the template dial sub-graph under the angle for each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results; n is a positive integer greater than 1.
For example, if the matching confidence degrees at 100 rotation angles are obtained through calculation, further, the matching confidence degrees at the 100 rotation angles are sorted from large to small, and the image feature similarity between the dial sub-graph at the angle and the template dial sub-graph is calculated for each rotation angle corresponding to the first 10 matching confidence degrees, so as to obtain 10 feature similarity results.
It can be understood that the result of selecting the first n matching confidences may be set systematically or manually, and the present invention does not limit the specific values of the first n.
S205, determining a pointer identification angle of the pointer instrument according to the maximum value of the n feature similarity results, and converting the pointer identification angle into a reading value according to instrument scale information.
In the embodiment of the present invention, the meter scale information may refer to a prestored comparison table in which each angle interval corresponds to one reading value, and different types of dial scale information have corresponding reading values, for example, a pointer of an electric meter corresponds to 120kwh in an interval of 30-40 degrees.
Illustratively, the server takes the maximum value of the 10 calculated feature similarity results, the angle of the dial plate image corresponding to the maximum value is determined as the pointer identification angle of the pointer instrument to be identified, and further, the pointer identification angle can be converted into a reading value according to the instrument scale information and displayed to the user for viewing.
Therefore, the invention carries out identification and calculation processing through a machine, not only does not need manual calculation, has less calculation amount but also has high accuracy, but also is suitable for reading identification of multi-scale, multi-dial and multi-pointer type instruments, and can be suitable for various scenes.
Optionally, the method further includes: acquiring a training data set, wherein the training data set comprises a dial image, a dial area, a corresponding dial type and the number of pointers in the dial; training a target detection model according to the training data set; correspondingly, inputting the input image into a pre-trained target detection model, detecting an instrument dial area in the input image, and obtaining a dial subgraph, wherein the steps of: inputting the input image into a target detection model obtained through training of the training data set, detecting an instrument dial area in the input image to obtain a dial subgraph, and determining the type of a dial and the number of pointers; and determining a template dial chart according to the dial type and the number of the pointers.
In the embodiment of the invention, a training data set provided manually is obtained, wherein the training data set can comprise various dial plate images, various dial plate areas, various corresponding dial plate types and the number of pointers in the dial plates; wherein, the number of the hands in the dial can be one or more.
It should be noted that the training of the target detection model by acquiring the training data set only needs to be performed once, and the trained target detection model can be directly used to identify the input image afterwards.
Illustratively, the server may train the target detection model in advance, that is, obtain a training data set, which may include various dial plate images, various dial plate regions, corresponding various dial plate types, and the number of pointers in the dial plates; further, the target detection model is trained according to the training data set. For example, the server can acquire data of various types of pointer instruments such as dial images, dial areas and the number of pointers in dials of different shapes of electric meters, dial images, dial areas and the number of pointers in dials of different shapes of water meters, and then train the target detection model according to the data.
Correspondingly, after the target detection model is trained, the server can input the input image 1 shot by the camera into the trained target detection model, so that the instrument dial area in the input image 1 can be detected, a required dial sub-graph can be obtained, the dial type can be determined to be an ammeter, and the number of pointers is 2; further, the template dial chart can be determined according to the type of the electric meter and the number of 2 pointers.
Therefore, the dial sub-graph is obtained by detecting the meter dial area in the input image by using the pre-trained target detection model, so that the detection accuracy can be improved, and the target detection model can identify various types of dials and has wide application range.
Optionally, the method further includes: normalizing the size of the meter dial area to the size of a template dial map, and/or normalizing the size of a pointer in the meter dial area to the size of a pointer in the template dial map; the template dial image is an image with preset size.
In the embodiment of the invention, normalization is a simplified calculation mode, and the absolute value of the physical system value can be changed into a certain relative value relationship.
The template dial map may be an image with a preset size, which may be a system-set size or an artificially-set size, and specific values of the size are not limited in the present invention, and may be, for example, 640 × 480 dpi.
For example, since the distances from the camera to the pointer instrument are not the same, the sizes of the shot images are different, and therefore the shot input image needs to be normalized to the size of the template chart, comparison calculation is facilitated, and calculation accuracy is improved. Therefore, the server may normalize the size of the meter dial area of the captured input image to the size of the template dial map, and may normalize the size of the pointer in the meter dial area of the captured input image to the size of the pointer in the template dial map, which is not particularly limited by the present invention.
It can be understood that the server can also normalize the size of the instrument dial area of the shot input image to the size of the template dial map, and also normalize the size of the pointer in the instrument dial area of the shot input image to the size of the pointer in the template dial map, so that the calculation accuracy is improved, and the reliability is high.
Optionally, the matching of the dial sub-graph and the template dial sub-graph by rotation includes: extracting a gray level image of the dial sub-graph, and acquiring a gray level template of the template dial sub-graph; and rotating the gray level image of the dial sub-graph by 0-360 degrees to match with the gray level template, or rotating the gray level template by 0-360 degrees to match with the gray level image of the dial sub-graph.
In an embodiment of the present invention, the grayscale image may be obtained by measuring the brightness of each pixel in a single electromagnetic spectrum (e.g., visible light), and may refer to an image represented by 256-level grayscale, and the grayscale template may refer to an image obtained by performing grayscale processing on a template dial map.
Illustratively, the server can extract the gray level image of the dial subgraph detected by the pre-trained target detection model, and can also obtain the gray level template of the template dial graph; furthermore, the obtained gray level images of the dial sub-images can be rotated clockwise every 1 degree for 360 degrees in total to obtain 360 images, the 360 images are respectively matched with the gray level templates, the images in the gray level templates can also be rotated clockwise every 1 degree for 360 degrees in total, and the obtained images are respectively matched with the gray level images of the dial sub-images.
Therefore, the gray level image of the dial sub-graph is matched with the gray level template, so that the matching calculation can be performed by using less data, and the calculation amount is reduced.
Optionally, calculating the matching confidence degrees at a plurality of rotation angles includes: acquiring a first local feature of the dial sub-graph and a second local feature of the template dial sub-graph; the first local feature and the second local feature are both image features of the region where the pointer is located; calculating a confidence of a match of the second local feature to the first local feature at a plurality of rotation angles.
In this embodiment of the present invention, the first local feature may refer to an image feature of an area where a pointer is located in an image of the dial sub-graph, and the second local feature may refer to an image feature of an area where a pointer is located in an image of the template dial sub-graph.
For example, the server may obtain a first local feature of a dial sub-image and a second local feature of a template dial sub-image detected by a pre-trained target detection model, that is, image features of regions where pointers of the two images are located, further extract pixel points of the image features of the regions where the pointers of the two images are located, and calculate matching confidence degrees of the second local feature and the first local feature at a plurality of rotation angles by using the extracted pixel points of the first local feature and the second local feature.
It can be understood that the region where the pointer is located is the necessary image feature for acquisition, and therefore, by calculating the matching confidence of the image features of the region where the pointer is located at a plurality of rotation angles, the calculation amount can be reduced while the accuracy is improved.
Optionally, the image feature is a feature of a color image; calculating the image feature similarity of the dial sub-graph and the template dial sub-graph at each angle in the rotation angles corresponding to the first n matching confidences, wherein the image feature similarity comprises the following steps: acquiring the characteristics of the template dial images corresponding to the first n matching confidence degrees and the color images of the dial sub-images, which are extracted by an image characteristic extractor, wherein the image characteristic extractor is an image characteristic extractor trained on the basis of a deep learning model; and calculating the similarity of the characteristics of the color image of the template dial chart corresponding to the first n matching confidences and the characteristics of the color image of the dial sub-chart.
In the embodiment of the present invention, the feature of the color image is a global feature, which is used to describe the surface property of the scene corresponding to the image or the image area. The image feature extractor can extract image information based on a computer and can perform high-level processing on digital images, and the image feature extractor of the embodiment of the invention is an image feature extractor trained based on a deep learning model, so that the extraction accuracy can be improved.
For example, if the server calculates and obtains matching confidence degrees at 100 rotation angles and performs size arrangement on the matching confidence degrees, the server may further obtain features of the color images of the template dial plate map and the dial plate sub-map corresponding to the first 10 matching confidence degrees extracted by the image feature extractor, and may calculate similarities between the features of the color images of the template dial plate map corresponding to the first 10 matching confidence degrees and the features of the color images of the dial plate sub-map.
Therefore, the feature similarity of the first n color images can be calculated, the recognition accuracy can be ensured to be high, and the calculation amount can be controlled to be small.
It is to be understood that, the similarity of the features of the color image may be calculated instead of calculating the similarity of various other image features, for example, the similarity of Histogram of Oriented Gradient (HOG) features, the similarity of scale-invariant feature transform (SIFT) features, and the like, which is not specifically limited in this embodiment of the present invention.
With reference to the foregoing embodiment, fig. 3 is a flowchart of a reading identification method for a pointer instrument according to an embodiment of the present invention. As shown in fig. 3, the method executed by the embodiment of the present invention includes the steps of:
step A: and (4) starting reading identification of the pointer instrument, shooting various dial plate images of various types according to a certain preset period, sequentially detecting the input various dial plate images based on a target detection model for deep learning, and executing the step B.
And B: and (4) adapting various types of dial plate images in a multi-scale mode, namely performing normalization processing on the various dial plate images to obtain a plurality of dial plate subgraphs and executing the step C.
And C: and D, carrying out rotation matching on the plurality of dial sub-graphs and the template dial graph with the plurality of pointers, calculating matching confidence degrees under a plurality of rotation angles, and executing the step D.
Step D: and E, sequencing the matching confidence degrees under the rotation angles according to a descending order, selecting the dial sub-images in the rotation angles corresponding to the first n matching confidence degrees and the images of the template dial sub-images, calculating the feature similarity of the images, obtaining n feature similarities, and executing the step E.
Step E: and selecting the pointer identification angle with the maximum similarity, converting the pointer identification angle into a reading, and finishing the reading identification of the pointer instrument.
In the foregoing embodiments, the pointer instrument reading identification method provided by the embodiments of the present invention is described, and in order to implement each function in the method provided by the embodiments of the present invention, the electronic device serving as the execution subject may include a hardware structure and/or a software module, and implement each function in the form of a hardware structure, a software module, or a hardware structure and a software module. Whether any of the above functions is implemented as a hardware structure, a software module, or a combination of a hardware structure and a software module depends upon the particular application and design constraints imposed on the technical solution.
For example, fig. 4 is a schematic structural diagram of a pointer instrument reading identification apparatus according to an embodiment of the present invention, and as shown in fig. 4, the apparatus includes: an obtaining module 410, configured to obtain an input image of a dial of a pointer instrument; the detection module 420 is configured to input the input image into a pre-trained target detection model, and detect an instrument dial area in the input image to obtain a dial subgraph; the target detection model is a deep learning model constructed based on a neural network; the calculating module 430 is used for performing rotation matching on the dial sub-graph and the template dial graph and calculating matching confidence coefficients under a plurality of rotation angles; the processing module 440 is configured to sort the matching confidence degrees under the multiple rotation angles in a descending order, and calculate, for each angle of the rotation angles corresponding to the first n matching confidence degrees, an image feature similarity between the dial sub-graph and the template dial sub-graph under the angle, so as to obtain n feature similarity results; n is a positive integer greater than 1; the determining module 450 is configured to determine a pointer identification angle of the pointer instrument according to a maximum value of the n feature similarity results, and convert the pointer identification angle into a reading value according to instrument scale information.
Optionally, the obtaining module 410 is further configured to obtain a training data set, where the training data set includes a dial image, an area where the dial is located, a corresponding dial type, and a number of pointers in the dial; the detection module 420 is further configured to train a target detection model according to the training data set; correspondingly, the detection module 420 is specifically configured to input the input image into a target detection model obtained through training of the training data set, detect an instrument dial area in the input image, obtain a dial subgraph, and determine a dial type and a pointer number; and determining a template dial chart according to the dial type and the number of the pointers.
Optionally, the processing module 440 is further configured to normalize the size of the meter dial area to the size of the template dial map, and/or normalize the size of the pointer in the meter dial area to the size of the pointer in the template dial map; the template dial image is an image with preset size.
Optionally, the calculating module 430 is specifically configured to extract the grayscale image of the dial sub-graph, and obtain a grayscale template of the template dial sub-graph; and rotating the gray level image of the dial sub-graph by 0-360 degrees to match with the gray level template, or rotating the gray level template by 0-360 degrees to match with the gray level image of the dial sub-graph.
Optionally, the calculating module 430 is specifically configured to obtain a first local feature of the dial sub-graph and a second local feature of the template dial sub-graph; the first local feature and the second local feature are both image features of the area where the pointer is located; calculating a confidence of the matching of the second local feature to the first local feature at a plurality of rotation angles.
Optionally, the image feature is a feature of a color image; the processing module 440 is specifically configured to obtain features of the template dial images and the color images of the dial sub-images, which are extracted by an image feature extractor, and correspond to the first n matching confidence degrees, where the image feature extractor is an image feature extractor trained based on a deep learning model; and calculating the similarity between the characteristics of the color image of the template dial plate image corresponding to the first n matching confidences and the characteristics of the color image of the dial plate sub-image.
Optionally, the meter scale information is a prestored comparison table in which each angle interval corresponds to a reading value.
An embodiment of the present invention further provides a schematic structural diagram of an electronic device, and fig. 5 is a schematic structural diagram of an electronic device provided in an embodiment of the present invention, and as shown in fig. 5, the electronic device may include: a memory 501 and a processor 502; the memory 501 stores a computer program; the processor 502 executes the computer program stored in the memory 501, so that the processor 502 executes the method according to any of the above embodiments.
The memory 501 and the processor 502 may be connected by a bus 503.
The specific implementation principle and effect of the pointer instrument reading identification device provided by the embodiment of the invention can be referred to the corresponding relevant description and effect of the above embodiment, and will not be described in detail herein.
Embodiments of the present invention further provide a computer-readable storage medium, where a computer program execution instruction is stored, and when the computer program execution instruction is executed by a processor, the computer program execution instruction is used to implement the pointer instrument reading identification method in any one of the foregoing embodiments of the present invention.
The embodiment of the invention also provides a chip for operating the instruction, wherein the chip is used for executing the pointer instrument reading identification method executed by the electronic equipment in any one of the embodiments of the invention.
Embodiments of the present invention further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for identifying reading of a pointer instrument performed by an electronic device according to any of the foregoing embodiments of the present invention can be implemented.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or modules, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to implement the solution of the embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor to execute some steps of the methods according to the embodiments of the present invention.
It should be understood that the processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may include a Random Access Memory (RAM), and may further include a non-volatile memory (NVM), such as at least one magnetic disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic disk or an optical disk.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present invention are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
The above description is only a specific implementation of the embodiments of the present invention, but the scope of the embodiments of the present invention is not limited thereto, and any changes or substitutions within the technical scope disclosed by the embodiments of the present invention should be covered within the scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A reading identification method for a pointer instrument is characterized by comprising the following steps:
acquiring an input image of a dial plate of the pointer instrument;
inputting the input image into a pre-trained target detection model, and detecting an instrument dial area in the input image to obtain a dial subgraph; the target detection model is a deep learning model constructed based on a neural network;
carrying out rotation matching on the dial sub-graph and the template dial graph, and calculating matching confidence coefficients under a plurality of rotation angles;
sequencing the matching confidence degrees under the plurality of rotation angles according to a descending order, and calculating the image feature similarity of the dial sub-graph and the template dial sub-graph under each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results; n is a positive integer greater than 1;
and determining a pointer identification angle of the pointer instrument according to the maximum value in the n characteristic similarity results, and converting the pointer identification angle into a reading value according to instrument scale information.
2. The method of claim 1, further comprising:
acquiring a training data set, wherein the training data set comprises a dial image, a dial area, a corresponding dial type and the number of pointers in the dial;
training a target detection model according to the training data set;
correspondingly, inputting the input image into a pre-trained target detection model, detecting an instrument dial area in the input image, and obtaining a dial sub-graph, including:
inputting the input image into a target detection model obtained through training of the training data set, detecting an instrument dial area in the input image to obtain a dial subgraph, and determining the type of a dial and the number of pointers;
and determining a template dial chart according to the dial type and the number of the pointers.
3. The method of claim 1, further comprising:
normalizing the size of the meter dial area to the size of a template dial map, and/or normalizing the size of a pointer in the meter dial area to the size of a pointer in the template dial map;
the template dial image is an image with preset size.
4. The method of claim 1, wherein rotationally matching the dial subgraph with the template dial graph comprises:
extracting the gray level image of the dial sub-graph, and acquiring a gray level template of the template dial sub-graph;
and rotating the gray level image of the dial sub-graph by 0-360 degrees to match with the gray level template, or rotating the gray level template by 0-360 degrees to match with the gray level image of the dial sub-graph.
5. The method of claim 1, wherein calculating the confidence of the match at a plurality of rotation angles comprises:
acquiring a first local feature of the dial sub-graph and a second local feature of the template dial sub-graph; the first local feature and the second local feature are both image features of the area where the pointer is located;
calculating a confidence of a match of the second local feature to the first local feature at a plurality of rotation angles.
6. The method of claim 1, wherein the image features are features of a color image; calculating the image feature similarity of the dial sub-graph and the template dial sub-graph at each angle in the rotation angles corresponding to the first n matching confidences, wherein the image feature similarity comprises the following steps:
acquiring the characteristics of the template dial images corresponding to the first n matching confidence degrees and the color images of the dial sub-images, which are extracted by an image characteristic extractor, wherein the image characteristic extractor is an image characteristic extractor trained on the basis of a deep learning model;
and calculating the similarity between the characteristics of the color image of the template dial plate image corresponding to the first n matching confidences and the characteristics of the color image of the dial plate sub-image.
7. The method according to any one of claims 1 to 6, wherein the meter scale information is a pre-stored look-up table with one reading per angle interval.
8. A pointer instrument reading identification device is characterized by comprising:
the acquisition module is used for acquiring an input image of the dial plate of the pointer instrument;
the detection module is used for inputting the input image into a pre-trained target detection model, detecting an instrument dial area in the input image and obtaining a dial subgraph; the target detection model is a deep learning model constructed based on a neural network;
the calculation module is used for carrying out rotation matching on the dial sub-graph and the template dial graph and calculating matching confidence coefficients under a plurality of rotation angles;
the processing module is used for sequencing the matching confidence degrees under the plurality of rotation angles from large to small, and calculating the image feature similarity of the dial sub-graph and the template dial sub-graph under each angle in the rotation angles corresponding to the first n matching confidence degrees to obtain n feature similarity results; n is a positive integer greater than 1;
and the determining module is used for determining the pointer identification angle of the pointer instrument according to the maximum value in the n characteristic similarity results and converting the pointer identification angle into a reading value according to instrument scale information.
9. An electronic device, comprising: a processor, a memory, and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the pointer meter reading identification method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions which, when executed by a processor, are used for implementing the pointer instrument reading identification method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111088339.8A CN115393838A (en) | 2021-09-16 | 2021-09-16 | Pointer instrument reading identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111088339.8A CN115393838A (en) | 2021-09-16 | 2021-09-16 | Pointer instrument reading identification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115393838A true CN115393838A (en) | 2022-11-25 |
Family
ID=84114649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111088339.8A Pending CN115393838A (en) | 2021-09-16 | 2021-09-16 | Pointer instrument reading identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115393838A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116189192A (en) * | 2023-04-24 | 2023-05-30 | 东方电子股份有限公司 | Intelligent reading identification method and system for pointer instrument |
-
2021
- 2021-09-16 CN CN202111088339.8A patent/CN115393838A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116189192A (en) * | 2023-04-24 | 2023-05-30 | 东方电子股份有限公司 | Intelligent reading identification method and system for pointer instrument |
| CN116189192B (en) * | 2023-04-24 | 2023-07-25 | 东方电子股份有限公司 | Intelligent reading identification method and system for pointer instrument |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107545241B (en) | Neural network model training and living body detection method, device and storage medium | |
| CN108182433B (en) | Meter reading identification method and system | |
| CN109977191B (en) | Problem map detection method, device, electronic equipment and medium | |
| Belan et al. | Segmentation-free approaches of computer vision for automatic calibration of digital and analog instruments | |
| CN106327546B (en) | Method and device for testing face detection algorithm | |
| CN112052186A (en) | Target detection method, device, equipment and storage medium | |
| CN113705462B (en) | Face recognition method, device, electronic equipment and computer readable storage medium | |
| CN114758249B (en) | Target object monitoring method, device, equipment and medium based on field night environment | |
| US20190138840A1 (en) | Automatic ruler detection | |
| CN114743189A (en) | Pointer instrument reading identification method and device, electronic equipment and storage medium | |
| CN110826646A (en) | Robot vision testing method and device, storage medium and terminal equipment | |
| CN112683169A (en) | Object size measuring method, device, equipment and storage medium | |
| CN109829383B (en) | Palmprint recognition method, palmprint recognition device and computer equipment | |
| Salomon et al. | Image-based automatic dial meter reading in unconstrained scenarios | |
| CN115393838A (en) | Pointer instrument reading identification method and device, electronic equipment and storage medium | |
| CN108875500A (en) | Pedestrian recognition methods, device, system and storage medium again | |
| CN114049568A (en) | Object shape change detection method, device, equipment and medium based on image comparison | |
| US10922569B2 (en) | Method and apparatus for detecting model reliability | |
| Liu et al. | An Edge-computing flow meter reading recognition algorithm optimized for agricultural IoT network | |
| CN115588196A (en) | Pointer type instrument reading method and device based on machine vision | |
| Zhang et al. | A YOLOv3‐Based Industrial Instrument Classification and Reading Recognition Method | |
| CN108898632B (en) | Method and device for determining rotation angle of instrument pointer | |
| Yang et al. | Real-time reading system for pointer meter based on YolactEdge | |
| CN114386156A (en) | BIM-based hidden member display method, device, equipment and readable storage medium | |
| CN114299509A (en) | Method, device, equipment and medium for acquiring information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |