CN111046881B - Pointer type instrument reading identification method based on computer vision and deep learning - Google Patents
Pointer type instrument reading identification method based on computer vision and deep learning Download PDFInfo
- Publication number
- CN111046881B CN111046881B CN201911219009.0A CN201911219009A CN111046881B CN 111046881 B CN111046881 B CN 111046881B CN 201911219009 A CN201911219009 A CN 201911219009A CN 111046881 B CN111046881 B CN 111046881B
- Authority
- CN
- China
- Prior art keywords
- image
- pointer
- area
- instrument
- line segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The pointer instrument reading identification method based on computer vision and deep learning comprises the following operation steps: (1) detecting the dial plate area of the pointer instrument; (2) preprocessing the pointer instrument image; (3) a scale and pointer detection process; (4) a digital area detection process; (5) a number identification process; (6) reading calculation process; the method can realize quick, accurate and automatic reading of the pointer instrument, and is time-saving and labor-saving.
Description
Technical Field
The invention relates to a pointer instrument reading identification method based on computer vision and deep learning, belongs to the technical field of computer vision, and particularly belongs to the technical field of detection and identification based on computer vision.
Background
At present, many pointer type instruments in China still need manual meter reading, so that time and labor are wasted, efficiency is low, and errors are easy to occur. Computer vision technology and deep learning technology have been greatly developed in recent years, and how to utilize the computer vision technology and the deep learning technology to realize automatic reading of instruments is a technical problem which is urgently needed to be solved by various industries.
Disclosure of Invention
In view of the above, the present invention is to provide a method based on computer vision and deep learning technology to realize automatic meter reading of a pointer instrument.
In order to achieve the above purpose, the invention provides a pointer instrument reading identification method based on computer vision and deep learning, which comprises the following operation steps:
(1) The specific content of the detection process of the dial plate area of the pointer instrument is as follows: inputting the pointer instrument image into a depth neural network trained in advance, detecting the instrument dial area, and returning the coordinate information of the pointer instrument dial area after detection;
(2) The image preprocessing process of the pointer instrument specifically comprises the following steps: dividing the instrument dial area image according to the instrument dial area coordinate information in the step (1); after division, the size of the instrument dial area image is processed in a unified way; performing expansion and corrosion operations on the uniformly processed standard image to obtain a preprocessed image of the dial area of the pointer instrument;
(3) The scale and pointer detection process comprises the following specific contents: extracting coordinate information of all line segments in the image according to the preprocessed image obtained in the step (2), and recording line segment angle information; calculating the vertical distance from the image center point to the straight line where the line segment is located, and calculating the straight line distance from the image center point to two end points of the line segment; judging whether each line segment is a scale line or a pointer according to a preset condition;
(4) The digital area detection process specifically comprises the following steps: rotating and dividing the preprocessed image in the step (2) according to the coordinate information and the angle information of the line segment judged as the scale mark returned in the step (3) to obtain an image of the area where the number corresponding to the scale is located; carrying out contour search on the digital area image to obtain single digital position information; further segmenting the single digital area image, and unifying the resolution of the segmented images;
(5) The digital identification process specifically comprises the following steps: identifying the single digital image returned in the step (4) through a pre-trained deep neural network to obtain a digit with the maximum confidence coefficient; sequencing the recognized numbers according to the single number position information obtained in the step (4) to obtain final numbers;
(6) The reading calculation process specifically comprises the following steps: calculating the value represented by each degree according to the angle information judged as the scale mark returned in the step (3) and the final number returned in the step (5); and (4) returning the angle information which is judged as the pointer according to the step (3) to calculate the final reading result.
The specific content of the coordinate information returned to the dial plate area of the pointer instrument after detection comprises the following operation steps:
(11) Selecting a deep neural network as a training network;
(12) Inputting the coordinates of the upper left corner and the lower right corner of the CLOCK data in the COCO data set as training time stamp data;
(13) The model output of the training network is set to 2 classes and 2 coordinates: the 2 categories are respectively non-instrument dials and the classification result is an instrument dial, and are 2 categories in total; the 2 coordinates are respectively the coordinates of the upper left corner of the meter dial area and the coordinates of the lower right corner of the meter dial area.
And (3) after the division in the step (2), uniformly processing the size of the instrument dial area image into a standard image with the height of 800 pixels and the width of 800 pixels.
The step (2) of performing expansion and corrosion operations on the uniformly processed standard image to obtain the specific content of the preprocessed image of the dial area of the pointer instrument comprises the following operation steps:
(21) Converting the uniformly processed dial plate image into a gray image;
(22) Performing expansion processing with convolution kernel of 3 on the gray level image;
(23) And performing corrosion treatment with convolution kernel of 3 twice on the image after the expansion treatment to obtain a preprocessed image.
The step (3) of extracting the coordinate information of all line segments in the image according to the obtained preprocessed image, and recording the specific content of the line segment angle information comprises the following operation steps:
(3101) Converting the preprocessed image into a binary image;
(3102) Calculating the binary image to obtain image edge information;
(3103) And extracting line segments according to the image edge information, and recording coordinates of two end points of the line segments and angles between the two end points and the horizontal direction.
The step (3) of judging whether each line segment is a scale line or the specific content of the pointer according to the preset conditions comprises the following operation steps:
(3201) Judging as a condition of the pointer: the distance from the image center point to the straight line where the line segment is located is less than m pixel points, the length of the line segment is greater than the straight line with the maximum length of n pixel points, and m and n are natural numbers;
(3202) Judging as a scale condition: the distance from the image center point to the straight line where the line segment is located is less than j pixel points, the length of the line segment is greater than that of the line segment of k pixels, and j and k are natural numbers.
The step (4) of rotating and dividing the preprocessed image according to the coordinate information and the angle information of the line segment determined as the scale mark, and acquiring the specific content of the image of the area where the number corresponding to the scale is located includes the following operation steps:
(41) Respectively calculating the distances from two end points of the scale mark to the central point of the image, taking a point with a smaller distance, and setting coordinates as (x, y);
(42) Rotating the image according to the angle information of the scale marks, wherein the rotation center is the midpoint of the image, the rotation angle is the angle of the scale marks minus 90 degrees, and the scale marks are vertical to the horizontal direction after rotation;
(43) And the image area is divided into an image area with the coordinates of (x, y-s) at the upper left corner and (x + t, y + s) at the lower right corner, wherein s and t are natural numbers.
In the step (4), the single digital area image is further segmented, and the images obtained after segmentation are processed into a standard image with the height of 72 pixels and the width of 72 pixels.
In the step (5), the returned single digital image is identified through a pre-trained deep neural network, and the specific content of the digital image with the maximum confidence coefficient is obtained through the following operation steps:
(5101) Selecting a deep neural network as a training network;
(5102) Ten numbers of ten thousand different fonts and 0 to 9 at different angles are generated by using an ImageFont library;
(5103) Respectively taking the generated images and the types of the images as the input and the labels of the network, and training the deep neural network;
(5104) Inputting the standard image subjected to unified processing in the step (4) into the trained deep neural network for prediction and recognition;
(5105) The model output of the network is confidence of 11 categories, and the 11 categories are respectively: the background category is non-digital images, and 0 to 9 ten digital categories, and the category with the highest confidence is selected as the output result of the deep neural network.
The step (5) of sequencing the identified numbers according to the returned acquired single number position information to obtain the specific content of the final number comprises the following operation steps:
(5201) Recording the coordinates of the upper left corner of the image judged as a number through the neural network and a judgment result;
(5202) Sorting the numbers according to the ascending x value of the coordinate at the upper left corner, and combining the sorted numbers into an integer;
(5203) If the number starts with 0, the result is transferred to the form of a few tenths.
The step (6) of calculating the specific content of the value represented by each degree according to the angle information judged as the scale mark returned in the step (3) and the final number returned in the step (5) comprises the following operation steps:
(6101) Each set of test results is represented by { final number: angle } into a dictionary;
(6102) Calculating the result of dividing the final number difference value of each two groups in the dictionary by the angle difference value to obtain the number represented by each degree;
(6103) Taking the median of all the numbers represented by each degree as the final result.
The specific content of calculating the final reading result according to the angle information returned as the judgment pointer in the step (6) comprises the following operation steps:
(6201) Each set in the final number dictionary is selected { final number: angle } to calculate a final reading;
(6202) Reading result = final number- (angle-pointer angle) × value represented per degree;
(6203) Taking the median of all reading results as the final reading result.
The invention has the advantages of realizing rapid, accurate and automatic reading of the pointer instrument, saving time and labor.
Drawings
FIG. 1 is a flow chart of a pointer instrument reading identification method based on computer vision and deep learning.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the accompanying drawings.
Referring to fig. 1, a pointer instrument reading identification method based on computer vision and deep learning proposed by the present invention is described, the method comprises the following operation steps:
(1) The specific content of the detection process of the dial plate area of the pointer instrument is as follows: inputting the pointer instrument image into a depth neural network trained in advance, detecting the instrument dial area, and returning the coordinate information of the pointer instrument dial area after detection;
(2) The image preprocessing process of the pointer instrument specifically comprises the following steps: dividing the instrument dial area image according to the instrument dial area coordinate information in the step (1); after division, the size of the instrument dial area image is processed in a unified way; performing expansion and corrosion operations on the uniformly processed standard image to obtain a preprocessed image of the dial area of the pointer instrument;
(3) The scale and pointer detection process comprises the following specific contents: extracting coordinate information of all line segments in the image according to the preprocessed image obtained in the step (2), and recording line segment angle information; calculating the vertical distance from the image center point to the straight line where the line segment is located, and calculating the straight line distance from the image center point to two end points of the line segment; judging whether each line segment is a scale line or a pointer according to preset conditions;
(4) The digital area detection process specifically comprises the following steps: rotating and dividing the preprocessed image in the step (2) according to the coordinate information and the angle information of the line segment judged as the scale mark returned in the step (3) to obtain an image of the area where the number corresponding to the scale is located; carrying out contour searching on the digital area image to obtain single digital position information; further segmenting the single digital area image, and unifying the resolution of the segmented images;
(5) The digital identification process specifically comprises the following steps: identifying the single digital image returned in the step (4) through a pre-trained deep neural network to obtain a digit with the maximum confidence coefficient; sequencing the identified numbers according to the single number position information obtained in the step (4) to obtain final numbers;
(6) The reading calculation process specifically comprises the following steps: calculating the value represented by each degree according to the angle information judged as the scale mark returned in the step (3) and the final number returned in the step (5); and (4) returning the angle information which is judged as the pointer according to the step (3) to calculate the final reading result.
The specific content of the coordinate information returned to the dial plate area of the pointer instrument after detection comprises the following operation steps:
(11) Selecting a deep neural network as a training network, and pre-training, wherein YOLOv3 under a Pythrch (the Pythrch is a python version of the torrech, is a neural network framework with a Facebook open source and is specially used for GPU accelerated deep neural network programming) framework is used as the training network in the implementation;
(12) Inputting the coordinates (x 1, y 1) of the upper left corner and the coordinates (x 2, y 2) of the lower right corner of CLOCK data in a COCO data set (the COCO data set is a large image data set published by Microsoft, is specially designed for object detection, segmentation, human body key point detection and semantic segmentation and covers 80 classes) as training time annotation data;
(13) The model output of the training network is set to 2 classes and 2 coordinates: the 2 categories are respectively non-instrument dials and the classification result is an instrument dial, and are 2 categories in total; the 2 coordinates are respectively the coordinates of the upper left corner of the dial area of the instrument and the coordinates of the lower right corner of the dial area of the instrument.
In the step (2), the size of the divided instrument dial area image is processed into a standard image with the height of 800 pixels and the width of 800 pixels by using a resize function in an OpenCV library.
The step (2) of performing expansion and corrosion operations on the uniformly processed standard image to obtain the specific content of the preprocessed image of the dial area of the pointer instrument comprises the following operation steps:
(21) Converting the uniformly processed dial plate image into a gray image by using a cvtColor function in OpenCV;
(22) Setting a convolution kernel of the gray image through a getStructuringElement function in OpenCV, and performing expansion processing with the convolution kernel being 3 by using an anode function in OpenCV;
(23) And setting a convolution kernel of the image after the expansion processing through a getStructuringElement function in OpenCV, and performing corrosion processing with the convolution kernel being 3 twice by using a partition function in OpenCV to obtain a preprocessed image.
The step (3) of extracting the coordinate information of all line segments in the image according to the obtained preprocessed image, and recording the specific content of the line segment angle information comprises the following operation steps:
(3101) Converting the preprocessed image into a binary image by using a cvtColor algorithm in OpenCV;
(3102) Calculating the binary image to obtain image edge information by using a Canny algorithm;
(3103) Extracting line segments by using a HoughLines algorithm in OpenCV according to the image edge information, and recording coordinates (x) of two end points of the line segments 1 ,y 1 )，(x 2 ,y 2 ) And an angle theta from the horizontal direction.
The step (3) of judging whether each line segment is a specific content of a scale line or a pointer according to a preset condition comprises the following operation steps:
(3201) Judging as a condition of the pointer: the distance from the image center point to the straight line of the line segment is less than m (m is 30 in the embodiment), and the length of the line segment is greater than the maximum length of n (n is 70 in the embodiment) pixel points;
(3202) Judging as a scale condition: the distance from the image center point to the straight line of the line segment is less than j (in the embodiment, j takes the value of 30) pixel points, and the length of the line segment is greater than the length of the line segment of k (in the embodiment, k takes the value of 40) pixel points.
Image center point (x) 0 ，y 0 ) The distance calculation formula to the straight line Ax + By + C =0 where the line segment is located is
The step (4) of rotating and dividing the preprocessed image according to the coordinate information and the angle information of the line segment determined as the scale mark, and acquiring the specific content of the image of the area where the number corresponding to the scale is located includes the following operation steps:
(41) Separately calculating two end points (x) of the scale mark 1 ,y 1 ),(x 2 ,y 2 ) To the image center point (x) 0 ,y 0 ) A distance of
Taking a point with a smaller distance, and setting the coordinate of the point as (x, y);
(42) According to the angle information theta of the scale mark, calculating a rotation matrix through a getTrotationmatrix 2D function in OpenCV, and rotating the image through an affine transformation function warpAffine, wherein the rotation center is an image center point, the rotation angle is theta-90 degrees, and the scale mark is vertical to the horizontal direction after rotation;
(43) The image segmentation method comprises the steps of segmenting an image area with the coordinates of (x, y-s) at the upper left corner and (x + t, y + s) at the lower right corner, wherein s is a natural number, and the value is 100 in the embodiment; t is a natural number, and takes 150 in the embodiment.
In the step (4), the single digital area image is further segmented, and the segmented image is uniformly processed into a standard image with the height of 72 pixels and the width of 72 pixels by using a resize function in an OpenCV library.
In the step (5), the returned single digital image is identified through a pre-trained deep neural network, and the specific content of the digital image with the maximum confidence coefficient is obtained through the following operation steps:
(5101) Selecting a deep neural network as a training network, and using a ResNet50 under a Pythrch frame as the training network in the embodiment;
(5102) Ten numbers of ten thousand different fonts and 0 to 9 at different angles are generated by using an ImageFont library in Python;
(5103) Respectively taking the generated images and the types of the images as the input and the labels of the network, and training the deep neural network;
(5104) Inputting the standard image subjected to unified processing in the step (4) into the trained deep neural network for prediction and recognition;
(5105) The model output of the network is confidence of 11 categories, and the 11 categories are respectively: the background category is non-digital images, and 0 to 9 ten digital categories, and the category with the highest confidence is selected as the output result of the deep neural network.
The step (5) of sequencing the identified numbers according to the returned acquired single number position information to obtain the specific content of the final number comprises the following operation steps:
(5201) Recording the coordinates (x, y) of the upper left corner of the image which is judged to be digital through the neural network and a judgment result;
(5202) And sorting the numbers according to the x value of the coordinate at the upper left corner from small to large, and combining the sorted numbers into an integer. For example, the image determined to be digital in step (5) is output as coordinates (340, 401) and classified as 5; coordinates (300, 400) classified as 1; coordinates (378, 397), classified as 0, then 300-340-378 in x-value order, then 1,5,0 in numerical order, and 150 in total;
(5203) If the number starts with 0, the result is transferred to the form of a few tenths. For example, the image determined to be digital in step (5) is output as coordinates (340, 401) and classified as 1; coordinates (378, 397), classified as 5; coordinates (300, 400), classified as 0, then 300-340-378 in x-value order from small to large, then 0,1,5 in numerical order, since starting with 0, the results go to 0.15.
The step of returning the angle information judged as the scale mark according to the step (3) and the final number returned in the step (5) and calculating the specific content of the numerical value represented by each degree comprises the following operation steps:
(6101) Each set of test results is represented by { final number: angle is added to the dictionary, such as a dictionary of 20:89, 40:151, 80:272};
(6102) Calculating the final number difference value of each two groups in the dictionary by dividing the result of the angle difference value to obtain the number represented by each degree, wherein the calculation results according to the dictionary are (20-40)/(89-151) =0.323, (20-80)/(89-272) =0.327, (40-80)/(151-272) =0.331 respectively;
(6103) Taking the median of all the numbers represented by each degree as the final result, calculating the median of 0.323,0.327,0.331 and 0.327 in the above results, and finally considering the value represented by each degree as 0.327.
The specific content of calculating the final reading result according to the angle information returned as the judgment pointer in the step (6) comprises the following operation steps:
(6201) Each set in the final number dictionary is selected { final number: angle to calculate the final reading result, such as a dictionary of 20:89, 40:151, 80:272};
(6202) Reading result = final number- (angle-pointer angle) × value represented by each degree, for example, the pointer angle is 160, and the reading results are calculated as 20- (89-160) × 0.327=43.2, 40- (151-160) × 0.327=42.9, and 80- (272-160) × 0.327=43.4, respectively;
(6203) Taking the median of all the reading results as the final reading result, and calculating the results according to the above to obtain 43.2, 42.9 and 43.4, wherein the median is 43.2, and finally determining that the final reading result is 43.2.
The inventor carries out a large number of experiments on the method, and the experimental results prove that the method is feasible and efficient.
Claims (12)
1. Pointer instrument reading identification method based on computer vision and deep learning, its characterized in that: the method comprises the following operation steps:
(1) The specific content of the detection process of the dial plate area of the pointer instrument is as follows: inputting the pointer instrument image into a depth neural network trained in advance, detecting the instrument dial area, and returning the coordinate information of the pointer instrument dial area after detection;
(2) The image preprocessing process of the pointer instrument specifically comprises the following steps: dividing the instrument dial area image according to the instrument dial area coordinate information in the step (1); after division, the size of the instrument dial area image is processed in a unified way; performing expansion and corrosion operations on the uniformly processed standard image to obtain a preprocessed image of the dial area of the pointer instrument;
(3) The scale and pointer detection process comprises the following specific contents: extracting coordinate information of all line segments in the image according to the preprocessed image obtained in the step (2), and recording line segment angle information; calculating the vertical distance from the image center point to the straight line where the line segment is located, and calculating the straight line distance from the image center point to two end points of the line segment; judging whether each line segment is a scale line or a pointer according to a preset condition;
(4) The digital area detection process specifically comprises the following steps: rotating and dividing the preprocessed image in the step (2) according to the coordinate information and the angle information of the line segment judged as the scale mark returned in the step (3) to obtain an image of the area where the number corresponding to the scale is located; carrying out contour search on the digital area image to obtain single digital position information; further segmenting the single digital area image, and unifying the resolution of the segmented images;
(5) The digital identification process specifically comprises the following steps: identifying the single digital image returned in the step (4) through a pre-trained deep neural network to obtain a digit with the maximum confidence coefficient; sequencing the identified numbers according to the single number position information obtained in the step (4) to obtain final numbers;
(6) The reading calculation process specifically comprises the following steps: calculating the value represented by each degree according to the angle information judged as the scale mark returned in the step (3) and the final number returned in the step (5); and (4) returning the angle information which is judged as the pointer according to the step (3) to calculate the final reading result.
2. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the specific content of the coordinate information returned to the dial plate area of the pointer instrument after detection comprises the following operation steps:
(11) Selecting a deep neural network as a training network;
(12) Inputting the coordinates of the upper left corner and the lower right corner of the CLOCK data in the COCO data set as training time stamp data;
(13) The model output of the training network is set to 2 classes and 2 coordinates: the 2 categories are respectively non-instrument dials and the classification result is an instrument dial, and are 2 categories in total; the 2 coordinates are respectively the coordinates of the upper left corner of the dial area of the instrument and the coordinates of the lower right corner of the dial area of the instrument.
3. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: and (3) after the division in the step (2), uniformly processing the size of the instrument dial area image into a standard image with the height of 800 pixels and the width of 800 pixels.
4. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (2) of performing expansion and corrosion operations on the uniformly processed standard image to obtain the specific content of the preprocessed image of the dial area of the pointer instrument comprises the following operation steps:
(21) Converting the uniformly processed dial plate image into a gray image;
(22) Performing expansion processing with convolution kernel of 3 on the gray level image;
(23) And performing corrosion treatment with convolution kernel of 3 twice on the image after the expansion treatment to obtain a preprocessed image.
5. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (3) of extracting the coordinate information of all line segments in the image according to the obtained preprocessed image, and recording the specific content of the line segment angle information comprises the following operation steps:
(3101) Converting the preprocessed image into a binary image;
(3102) Calculating the binary image to obtain image edge information;
(3103) And extracting line segments according to the image edge information, and recording coordinates of two end points of the line segments and angles between the two end points and the horizontal direction.
6. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (3) of judging whether each line segment is a specific content of a scale line or a pointer according to a preset condition comprises the following operation steps:
(3201) Judging as a condition of the pointer: the distance from the image center point to the straight line where the line segment is located is less than m pixel points, the length of the line segment is greater than the straight line with the maximum length of n pixel points, and m and n are natural numbers;
(3202) Judging as a scale condition: the distance from the image center point to the straight line where the line segment is located is less than j pixel points, the length of the line segment is greater than that of the line segment of k pixels, and j and k are natural numbers.
7. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (4) of rotating and dividing the preprocessed image according to the coordinate information and the angle information of the line segment determined as the scale mark, and acquiring the specific content of the image of the area where the number corresponding to the scale is located includes the following operation steps:
(41) Respectively calculating the distances from the two end points of the scale mark to the central point of the image, taking a point with a smaller distance, and setting the coordinates as (x, y);
(42) Rotating the image according to the angle information of the scale marks, wherein the rotation center is the midpoint of the image, the rotation angle is the angle of the scale marks minus 90 degrees, and the scale marks are vertical to the horizontal direction after rotation;
(43) And the image area is divided into an image area with the coordinates of (x, y-s) at the upper left corner and (x + t, y + s) at the lower right corner, wherein s and t are natural numbers.
8. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: in the step (4), the single digital area image is further segmented, and the images obtained after segmentation are processed into a standard image with the height of 72 pixels and the width of 72 pixels.
9. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: in the step (5), the returned single digital image is identified through a pre-trained deep neural network, and the specific content of the digital image with the maximum confidence coefficient is obtained through the following operation steps:
(5101) Selecting a deep neural network as a training network;
(5102) Ten numbers of ten thousand different fonts and 0 to 9 at different angles are generated by using an ImageFont library;
(5103) Respectively taking the generated images and the types of the images as the input and the labels of the network, and training the deep neural network;
(5104) Inputting the standard image subjected to unified processing in the step (4) into the trained deep neural network for prediction and recognition;
(5105) The model output of the network is confidence of 11 categories, and the 11 categories are respectively: the background category is non-digital images, and 0 to 9 ten digital categories, and the category with the highest confidence is selected as the output result of the deep neural network.
10. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (5) of sequencing the identified numbers according to the returned acquired single number position information to obtain the specific content of the final number comprises the following operation steps:
(5201) Recording the coordinates of the upper left corner of the image judged as a number through the neural network and a judgment result;
(5202) Sorting the numbers according to the x value of the coordinate at the upper left corner from small to large, and combining the sorted numbers into an integer;
(5203) If the number starts with 0, the result is transferred to the form of a few tenths.
11. The pointer instrument reading identification method based on computer vision and deep learning of claim 1, characterized in that: the step (6) of returning the angle information determined as the scale mark according to the step (3) and the final number returned in the step (5) and calculating the specific content of the value represented by each degree comprises the following operation steps:
(6101) Each set of test results is represented by { final number: angle } into a dictionary;
(6102) Calculating the result of dividing the final number difference value of each two groups in the dictionary by the angle difference value to obtain the number represented by each degree;
(6103) Taking the median of all numbers represented by each degree as the final result.
12. The pointer instrument reading identification method based on computer vision and deep learning of claim 1 or 11, wherein: the specific content of calculating the final reading result according to the angle information returned as the judgment pointer in the step (6) comprises the following operation steps:
(6201) Each set in the final number dictionary is selected { final number: angle } to calculate a final reading;
(6202) Reading result = final number- (angle-pointer angle) × value represented per degree;
(6203) Taking the median of all reading results as the final reading result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911219009.0A CN111046881B (en) | 2019-12-02 | 2019-12-02 | Pointer type instrument reading identification method based on computer vision and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911219009.0A CN111046881B (en) | 2019-12-02 | 2019-12-02 | Pointer type instrument reading identification method based on computer vision and deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111046881A CN111046881A (en) | 2020-04-21 |
| CN111046881B true CN111046881B (en) | 2023-03-24 |
Family
ID=70234289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911219009.0A Active CN111046881B (en) | 2019-12-02 | 2019-12-02 | Pointer type instrument reading identification method based on computer vision and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111046881B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598094B (en) * | 2020-05-27 | 2023-08-18 | 深圳市铁越电气有限公司 | Angle regression instrument reading identification method, equipment and system based on deep learning |
| CN112488099B (en) * | 2020-11-25 | 2022-12-16 | 上海电力大学 | Digital detection extraction element on electric power liquid crystal instrument based on video |
| CN112949564B (en) * | 2021-02-02 | 2022-11-29 | 电子科技大学 | Pointer type instrument automatic reading method based on deep learning |
| CN113435300B (en) * | 2021-06-23 | 2022-10-14 | 国网智能科技股份有限公司 | Real-time identification method and system for lightning arrester instrument |
| CN113609984A (en) * | 2021-08-05 | 2021-11-05 | 国网山东省电力公司德州市陵城区供电公司 | Pointer instrument reading identification method and device and electronic equipment |
| CN114037993B (en) * | 2021-09-26 | 2023-06-23 | 佛山中科云图智能科技有限公司 | Substation pointer instrument reading method and device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3998215B1 (en) * | 2007-03-29 | 2007-10-24 | 国立大学法人山口大学 | Image processing apparatus, image processing method, and image processing program |
| CN109344820A (en) * | 2018-08-06 | 2019-02-15 | 北京邮电大学 | Digital electric meter Recognition of Reading method based on computer vision and deep learning |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590498B (en) * | 2017-09-27 | 2020-09-01 | 哈尔滨工业大学 | Self-adaptive automobile instrument detection method based on character segmentation cascade two classifiers |
-
2019
- 2019-12-02 CN CN201911219009.0A patent/CN111046881B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3998215B1 (en) * | 2007-03-29 | 2007-10-24 | 国立大学法人山口大学 | Image processing apparatus, image processing method, and image processing program |
| CN109344820A (en) * | 2018-08-06 | 2019-02-15 | 北京邮电大学 | Digital electric meter Recognition of Reading method based on computer vision and deep learning |
| CN109543682A (en) * | 2018-11-23 | 2019-03-29 | 电子科技大学 | A kind of readings of pointer type meters method based on deep learning |
| CN109993166A (en) * | 2019-04-03 | 2019-07-09 | 同济大学 | The readings of pointer type meters automatic identifying method searched based on scale |
Non-Patent Citations (2)
| Title |
|---|
| 一种电力指针式仪表示数自动识别的鲁棒方法;佘世洲等;《计算机技术与发展》;20171205(第04期);全文 * |
| 基于机器视觉的指针式仪表示数识别方法研究;童伟圆等;《计算机测量与控制》;20180325(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111046881A (en) | 2020-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111046881B (en) | Pointer type instrument reading identification method based on computer vision and deep learning | |
| CN112949564B (en) | Pointer type instrument automatic reading method based on deep learning | |
| CN111626190B (en) | Water level monitoring method for scale recognition based on clustering partition | |
| CN106529537B (en) | A kind of digital instrument reading image-recognizing method | |
| WO2017016240A1 (en) | Banknote serial number identification method | |
| CN108921163A (en) | A kind of packaging coding detection method based on deep learning | |
| CN104700092B (en) | A kind of small characters digit recognition method being combined based on template and characteristic matching | |
| CN109426814B (en) | Method, system and equipment for positioning and identifying specific plate of invoice picture | |
| CN111539330B (en) | Transformer substation digital display instrument identification method based on double-SVM multi-classifier | |
| CN111523622B (en) | Method for simulating handwriting by mechanical arm based on characteristic image self-learning | |
| CN111222507B (en) | Automatic identification method for digital meter reading and computer readable storage medium | |
| CN112149667A (en) | Method for automatically reading pointer type instrument based on deep learning | |
| Mishchenko et al. | Chart image understanding and numerical data extraction | |
| CN113158895A (en) | Bill identification method and device, electronic equipment and storage medium | |
| CN110659637A (en) | Electric energy meter number and label automatic identification method combining deep neural network and SIFT features | |
| CN112734729A (en) | Water gauge water level line image detection method and device suitable for night light supplement condition and storage medium | |
| CN114241469A (en) | Information identification method and device for electricity meter rotation process | |
| Azizah et al. | Tajweed-YOLO: Object Detection Method for Tajweed by Applying HSV Color Model Augmentation on Mushaf Images | |
| CN111914706B (en) | Method and device for detecting and controlling quality of text detection output result | |
| CN111311602A (en) | Lip image segmentation device and method for traditional Chinese medicine facial diagnosis | |
| CN116994269A (en) | Seal similarity comparison method and seal similarity comparison system in image document | |
| CN114898347A (en) | Machine vision identification method for pointer instrument | |
| Mishchenko et al. | Model-Based Recognition and Extraction of Information from Chart Images. | |
| Sun et al. | Contextual models for automatic building extraction in high resolution remote sensing image using object-based boosting method | |
| JP2004094427A (en) | Slip image processor and program for realizing the same device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |