CN115546793B - Automatic reading method and system for water gauge scales and electronic equipment - Google Patents
Automatic reading method and system for water gauge scales and electronic equipment Download PDFInfo
- Publication number
- CN115546793B CN115546793B CN202211550233.XA CN202211550233A CN115546793B CN 115546793 B CN115546793 B CN 115546793B CN 202211550233 A CN202211550233 A CN 202211550233A CN 115546793 B CN115546793 B CN 115546793B
- Authority
- CN
- China
- Prior art keywords
- water gauge
- water
- letter
- image
- lowest number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/147—Determination of region of interest
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/15—Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/19007—Matching; Proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Abstract
The invention discloses a method, a system and electronic equipment for automatically reading water gauge scales, wherein the method comprises the following steps: acquiring a water gauge image to be identified; inputting the water gauge image to be recognized into a pre-trained character detector to obtain a character detection frame, and inputting the water gauge image to be recognized into a pre-trained scene divider to obtain a scene division image; and obtaining the lowest number of the water gauge and the relation between the surrounding letter E of the lowest number and the water surface according to the character detection frame and the scene segmentation image, and obtaining the water gauge reading according to the lowest number and the relation. The method can improve the accuracy of the scale reading of the water gauge and simultaneously ensure the reading efficiency.
Description
Technical Field
The invention relates to an image processing technology, in particular to a method and a system for automatically reading scales of a water gauge and electronic equipment.
Background
The water level is the most intuitive factor reflecting the water condition of the water body, the change of the water level is caused by the increase and decrease of the water quantity of the water body, the continuous and reliable water level monitoring has important significance for water resource scheduling and flood prevention and drought control, and the mastery degree of the water level information is also one embodiment of the water resource monitoring capability of the country. More and more water level collection detection points are provided in recent years, convenience is provided for real-time water level monitoring, and the water gauge records the height of the water level through reading, is the most intuitive and simple measuring tool and almost becomes the standard allocation of a hydrological station.
In recent years, video monitoring systems are built at a plurality of important water level observation points in China and are provided with standard water gauges, so that favorable conditions are provided for water level detection of the water gauges based on video images. The water gauge in the reservoir can be monitored by adopting an image method according to monitoring equipment, human eyes are replaced by image sensors to obtain water gauge images, and the reading corresponding to the water level line is detected by an image processing technology, so that water level information is automatically obtained.
At present, the method for identifying water level based on image processing mainly comprises: the method adopts the traditional image processing method, such as binaryzation processing, morphological analysis, corrosion expansion and other operations on the image. However, in a case where the environment around the water gauge is complicated or light is weak, the conventional image processing method introduces a lot of noise, which is disadvantageous to accurate water level identification. And secondly, identifying the information on the water gauge by adopting target detection based on the deep learning water gauge scale identification method. However, since the recognized letter E at the bottom of the water gauge may be immersed in water, the letter E is incomplete, and the real scale of the water gauge cannot be accurately recognized only by target detection.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, the invention aims to provide an automatic reading method for the water gauge scale, which can improve the reading accuracy of the water gauge scale and ensure the reading efficiency.
A second object of the present invention is to provide an electronic device.
The third purpose of the invention is to provide an automatic reading system for the water gauge scales.
In order to achieve the above object, an embodiment of the first aspect of the present invention provides an automatic reading method for a water gauge scale, including: acquiring a water gauge image to be identified; inputting the water gauge image to be recognized into a pre-trained character detector to obtain a character detection frame, and inputting the water gauge image to be recognized into a pre-trained scene divider to obtain a scene division image; and obtaining the lowest number of the water gauge according to the character detection frame and the scene segmentation image, the relation between the surrounding letters E of the lowest number and the water surface, and obtaining the water gauge reading according to the lowest number and the relation.
In addition, the automatic reading method for the water gauge scales provided by the embodiment of the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, the acquiring the water gauge image to be identified comprises the following steps: receiving byte information sent by a client through a target HTTP (hyper text transport protocol) website and a port number, wherein the byte information is obtained by encoding the water gauge image to be identified by the client; and decoding the byte information to obtain the water gauge image to be identified.
According to an embodiment of the invention, the method further comprises: and sending the water gauge reading to the client through the target HTTP website and the port number.
According to an embodiment of the present invention, the obtaining a lowest number of a water gauge according to the character detection box and the scene segmentation image, a relationship between a surrounding letter E of the lowest number and a water surface, and obtaining a water gauge reading according to the lowest number and the relationship includes: sequencing the number detection frames in the character detection frames to obtain a number character string, matching the number character string with a preset number string, and obtaining the lowest number according to a matching result; carrying out image fusion on the water surface area and the aquatic weed area in the scene segmentation image to obtain a water area; judging whether the water gauge is in water or not according to the water area and a character detection frame corresponding to a peripheral letter E of the lowest number; and if so, obtaining the scale grade of a target letter E in water, and obtaining the water gauge reading according to the lowest number and the scale grade, wherein the target letter E is the letter E in the surrounding letters E in water.
According to an embodiment of the invention, the method further comprises: before the character detection frames are sequenced, filtering out the character detection frames with the confidence degrees smaller than a preset confidence degree threshold value; before determining a water area according to the scene segmentation image, setting the pixel point of which the pixel value is greater than a preset pixel value confidence coefficient threshold value in the scene segmentation image as 255, and setting the pixel point of which the pixel value is less than or equal to the preset pixel value confidence coefficient threshold value as 0.
According to an embodiment of the invention, the method further comprises: before the water gauge image to be recognized is input into a pre-trained character detector, setting the size of the water gauge image to be recognized as 640 x 640; and before the water gauge image to be recognized is input into a pre-trained scene divider, setting the size of the water gauge image to be recognized as 512 x 512.
According to an embodiment of the present invention, the training process of the character detector and the scene segmenter includes: acquiring a training data set, wherein the training data set comprises a plurality of water gauge images and a first label and a second label of each water gauge image; constructing a character detector, and training the character detector by using a water gauge image in the training data set and a first label thereof, wherein the character detector adopts a YOLOv5 target detection network, and the first label comprises numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 and letters E in the water gauge image; and constructing a scene divider, and training the scene divider by using the water gauge image in the training data set and a second label thereof, wherein the scene divider adopts a UNet semantic division network, and the second label comprises a water gauge area, a water surface area, a float grass area and a background area in the water gauge image.
According to one embodiment of the invention, labelImg labeling software is adopted to label the numbers 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9 and the letter E on the water gauge image to obtain the first label; and labeling the water gauge region, the water surface region, the float grass region and the background region in the water gauge image by using labelme labeling software to obtain the second label.
To achieve the above object, an electronic device according to an embodiment of the second aspect of the present invention includes a memory, a processor, and a computer program stored in the memory, where the computer program is stored on the memory, and when the computer program is executed by the processor, the electronic device implements the automatic water scale reading method according to an embodiment of the first aspect of the present invention.
To achieve the above object, an automatic reading system for a water gauge scale according to a third aspect of the present invention comprises: a water gauge; the client is used for carrying out image acquisition on the water gauge and identifying a water gauge image; and the server is in communication connection with the client, and the server adopts the electronic equipment according to the embodiment of the second aspect of the invention.
According to the automatic reading method of the water gauge scale, the logic is simple, the reading efficiency is high, the detection capability of a small target during reading of the water gauge scale is improved by inputting the water gauge image to be recognized into the pre-trained character detector and the scene divider, the relation between the determined lowest number and the peripheral letter E of the lowest number and the water surface is more accurate, and the accuracy of the reading of the water gauge scale can be improved to a certain extent.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a schematic flow chart of a method for automatically reading a scale of a water gauge according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of step S101 in the automatic reading method for water gauge scales according to the embodiment of the present invention;
FIG. 3 is a schematic flow chart of a method for automatically reading a water gauge scale according to an embodiment of the invention;
fig. 4 is a schematic flowchart of step S103 in the automatic reading method for water gauge scales according to the embodiment of the present invention;
FIG. 5 is a schematic flow chart of a training process for a character detector and a scene divider in the automatic reading method for a water gauge scale according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the invention;
fig. 7 is a schematic structural diagram of an automatic reading system for a water gauge scale according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The automatic reading method, system and electronic device for the water gauge scales according to the embodiments of the present invention are described below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of an automatic reading method for a water gauge scale according to an embodiment of the invention.
As shown in fig. 1, in some embodiments, the automatic reading method of the water gauge scale may include the following steps:
and S101, acquiring a water gauge image to be identified.
Illustratively, the water gauge image to be identified may be acquired by a video monitoring system established at the water level observation point, that is, the water gauge in the reservoir is monitored according to the monitoring device, and the image of the water gauge to be identified in this embodiment is acquired by the image sensor.
S102, inputting the water gauge image to be recognized into a pre-trained character detector to obtain a character detection frame, and inputting the water gauge image to be recognized into a pre-trained scene divider to obtain a scene division image.
It is to be understood that the water gauge image to be recognized is input into a pre-trained character detector to obtain information of each number and character detection frame, and the water gauge image to be recognized is input into a pre-trained scene divider to obtain a scene divided image which is a binary image. The binary image refers to that gray values of pixel points on the image are respectively set to be 0 and 255, that is, the color image is changed into a black-and-white image, each pixel of the binary image has only two possible forms, namely pure black or pure white, and when the image is analyzed and processed, the contrast can be more obvious to a certain degree, and meanwhile, the interference of redundant pixels of the color image can be removed.
It should be noted that, in the embodiment of the present invention, the implementation manner of the pre-training work of the character detector and the scene segmenter may be selected according to historical experience of related people or actual requirements, and is not specifically limited in the embodiment of the present invention.
And S103, dividing the image according to the character detection frame and the scene to obtain the lowest number of the water gauge, the relation between the surrounding letters E of the lowest number and the water surface, and obtaining the reading of the water gauge according to the lowest number and the relation.
Specifically, after the corresponding character detection frame and scene segmentation image are obtained according to the water gauge image to be recognized in step S102, the related character detection frame information and scene segmentation image information can be obtained, the lowest number detection frame can be determined by sorting the character detection frames, the lowest number of the water gauge and the coordinate position information of the letter E frame around the number (namely, the relation between the letter E around the lowest number and the water surface) can be obtained, and the water gauge reading can be obtained by analyzing the relation between the surrounding letters E of the lowest number and the lowest number of the water gauge and the water surface.
Compared with the prior art, the automatic reading method for the water gauge scale provided by the embodiment of the invention has the advantages that the water gauge image to be recognized is input into the pre-trained character detector to obtain the character detection frame, so that the reading error of the water gauge caused by the false detection of the detection frame can be reduced, the water gauge image to be recognized is input into the pre-trained scene divider to obtain the scene division image, the relatively accurate relation between the water gauge and the water surface can be obtained, the accuracy of the obtained water gauge scale reading is improved, and the reliability of the automatic reading work of the water gauge scale is ensured.
As a possible implementation manner, as shown in fig. 2, in step S101 in the foregoing embodiment, acquiring a water gauge image to be recognized may include:
s201, receiving byte information sent by a client through a target HTTP website and a port number, wherein the byte information is obtained by encoding a to-be-identified water gauge image by the client.
And S202, decoding the byte information to obtain a water gauge image to be identified.
That is to say, the client encodes the water gauge image to be identified into bytes and sends the bytes to the designated HTTP website and port number, the server can obtain the byte information corresponding to the image under the corresponding HTTP website and port number, and the water gauge image to be identified can be obtained by decoding and recovering the byte information.
In some embodiments of the present invention, after obtaining the water gauge reading, the water gauge reading needs to be sent to the client, as shown in fig. 3, the method for automatically reading the water gauge scale may include:
and S301, acquiring a water gauge image to be identified.
S302, inputting the water gauge image to be recognized into a pre-trained character detector to obtain a character detection frame, and inputting the water gauge image to be recognized into a pre-trained scene divider to obtain a scene division image.
And S303, according to the character detection frame and the scene segmentation image, obtaining the lowest number of the water gauge, the relation between the surrounding letters E of the lowest number and the water surface, and according to the lowest number and the relation, obtaining the water gauge reading.
It should be noted that, for the specific implementation of steps S301 to S303, reference may be made to the implementation processes of steps S101 to S103 in the foregoing embodiments, and details are not described here for the purpose of reducing redundancy.
And S304, sending the water gauge reading to the client through the target HTTP website and the port number.
That is to say, after obtaining the water gauge reading, the server may send the scale information of the water gauge to the corresponding HTTP website and port number (that is, the HTTP website and port number for receiving the byte information sent by the client in the foregoing implementation manner) in the form of a character string from the water gauge reading, and then the client may obtain the water gauge reading result. Understandably, the scale information of the water gauge is sent to the corresponding HTTP website and port number in a character string mode according to the water gauge reading, so that the client can call the water gauge reading at any time, and convenience is brought to further data analysis.
As a possible implementation manner, in an embodiment of the automatic reading method for the water gauge scale, as shown in fig. 4, obtaining a relation between a bottommost number of the water gauge and surrounding letters E of the bottommost number and a water surface according to the character detection box and the scene segmentation image, and obtaining a water gauge reading according to the bottommost number and the relation may include:
s401, sequencing the number detection boxes in the character detection boxes to obtain a number character string, matching the number character string with a preset number string, and obtaining the lowest number according to a matching result.
The method for sorting the number detection boxes in the character detection boxes can be from small to large or from large to small, and the method can be selected according to specific requirements in practical application, and is not particularly limited in the embodiment of the invention. To clarify the matching process, a large-to-small example is given next, that is, the number detection boxes in the character detection boxes are sorted from large to small to obtain corresponding number character strings, the number character strings are divided into a plurality of character strings, the preset number strings are set as [9,8,7,6,5,4,3,2,1,0], character string matching is performed on the divided character strings and the preset number strings, the divided character strings are used as pattern strings, the preset number strings are used as main strings, and whether the pattern strings exist in the main strings is searched.
Optionally, in some embodiments, the numeric character string and the preset numeric string are matched through a kmp (Knuth-Morris-Pratt) algorithm, that is, internal matching information of the pattern string (i.e., the plurality of strings separated from the numeric character string) is calculated in advance, including all prefixes and suffixes of the separated plurality of strings, a maximum length table of common elements of corresponding prefix suffixes is calculated, a maximum length value, that is, a longest string that is the most matched is calculated according to the maximum length table, and finally, the lowest number is obtained.
S402, carrying out image fusion on the water surface area and the float grass area in the scene segmentation image to obtain a water area.
It can be understood that the obtained scene segmentation image may include a background area and a water gauge area in addition to the water surface area and the water grass area, and when the water area needs to be determined, the background area and the water gauge area do not need to be fused, and only the water surface area and the water grass area need to be subjected to image fusion.
And S403, judging whether the water gauge is in the water according to the water area and the character detection frame corresponding to the peripheral letter E of the lowest number.
As a feasible implementation manner, when judging whether the water gauge is in water according to the water area and the character detection frame corresponding to the peripheral letter E of the lowest number, the width and the height of the character detection frame corresponding to the letter E on the lower side of the lowest number can be firstly obtained, the intersection of the character detection frame and the water surface is taken, if the character detection frame and the water surface do not have the intersection, the returned water gauge result is-1, and the water gauge can be judged to be totally hung on the water surface; and if the character detection frame and the water surface have intersection, judging that the water gauge is in the water.
And S404, if so, obtaining the scale grade of the target letter E in the water, and obtaining the water gauge reading according to the lowest number and the scale grade, wherein the target letter E is the letter E in the surrounding letters E in the water.
It can be understood that, since the surrounding letters E of the lowermost number of the water gauge may include a plurality of letters, when it is determined that the water gauge is in the water according to the step S403, when determining the reading of the water gauge, the scale level of the letter E in the water in the surrounding letters E of the lowermost number in the water needs to be determined first, and then the calculation of the reading of the water gauge is performed according to the scale level and the lowermost number. And when the determined scale grades are different, the mode of calculating the water gauge reading can be different.
As an example, when determining the scale level of the target letter E in water and obtaining a water gauge reading based on the lowest number and scale level, may include:
and A, judging whether the right letter E of the lowest digit is detected.
And B, if the detection frame is detected, performing character grade estimation according to the detection frame corresponding to the right letter E of the lowest number, otherwise, calculating the distance between the lowest number and the water surface, and performing character grade estimation on the distance to obtain the height value of the detection frame corresponding to the right letter E of the lowest number.
Specifically, when the bottom digital right letter E is detected, the median of the heights of the detection boxes corresponding to all the letters E on the water gauge needs to be determined first, the ratio of the height of the detection box corresponding to the detected bottom digital right letter E to the median of the heights of the detection boxes corresponding to all the letters E on the water gauge is calculated, the corresponding character grade is determined according to the calculated ratio, and the height value of the detection box corresponding to the corresponding bottom digital right letter E is determined according to the character grade.
As an example, the height value of the detection box corresponding to the lowermost numeric right letter E can be determined by:
It should be understood that the distance between the lowest number and the water surface is the height of the character detection box corresponding to the letter E on the lower side of the lowest data minus the height of the intersection of the character detection box and the water surface. If the letter E on the right side of the lowest number is not detected, the ratio of the distance to the median of the heights of the detection frames corresponding to all the letters E on the water gauge can be calculated, and the character grade estimation work can be carried out according to the ratio.
And C, judging whether the height value of the detection frame corresponding to the right letter E of the lowest digit is smaller than a preset height.
D, if yes, determining the scale grade of the target letter E in the water as the grade of the letter E on the right side of the lowest number, and adding the lowest number to the height value of the detection frame corresponding to the letter E to obtain the final water gauge reading; otherwise, judging whether the lower letter E of the lowest number is detected.
And E, if the detection result is detected, performing character grade estimation according to the detection frame corresponding to the lower letter E of the lowest number, obtaining a ratio, and judging whether the ratio is close to the height of the detection frame corresponding to the lower letter E of the lowest number.
And F, if so, calculating the distance between the detection frame corresponding to the lower letter E of the lowest number and the water surface, and performing character grade estimation according to the distance and the ratio of the character detection frames, otherwise, performing character grade estimation according to the ratio obtained by performing character grade estimation on the detection frame corresponding to the lower letter E of the lowest number, and obtaining the height value of the detection frame corresponding to the lower letter E of the lowest number.
And F, determining the scale grade of the target letter E in the water as the grade of the letter E under the lowest number, and subtracting the height value of the detection frame corresponding to the letter E from the lowest number to obtain the final water gauge reading.
It should be noted that the manner of obtaining the water gauge reading according to the lowest number and the scale level shown above is only an example, and the calculation manner in the specific determination process is not unique, and may be adapted according to the historical experience of the relevant person or the actual application requirement, and no specific limitation is made in the embodiment of the present invention.
In some embodiments of the present invention, the method for automatically reading the water gauge scale may further include: before sorting the character detection frames, filtering out the character detection frames with the confidence degrees smaller than a preset confidence degree threshold value; before determining a water region according to a scene segmentation image, setting a pixel point with a pixel value larger than a preset pixel value confidence coefficient threshold value in the scene segmentation image as 255, and setting a pixel point with a pixel value smaller than or equal to the preset pixel value confidence coefficient threshold value as 0.
It can be understood that the preset confidence threshold is set to filter out some character detection frames with low confidence, and the preset pixel value confidence threshold is set to reduce some false-detection pixel points, which can improve the reliability of the automatic reading operation of the embodiment of the invention.
The preset confidence threshold and the preset pixel value confidence threshold may be determined according to actual conditions or historical experiences of related personnel, for example, the preset pixel value confidence threshold is set to 0.5, whether the confidence of the pixel point of the predicted image is greater than 0.5 is determined, the pixel point with the pixel value confidence greater than 0.5 is set to 255, and the pixel point with the pixel value confidence less than 0.5 is set to 0.
In some embodiments of the present invention, the method for automatically reading the water gauge scale may further comprise: before the water gauge image to be recognized is input into a pre-trained character detector, setting the size of the water gauge image to be recognized as 640 x 640; before the water gauge image to be recognized is input into the pre-trained scene divider, the size of the water gauge image to be recognized is set to 512 × 512.
It can be understood that before the water gauge image to be recognized is input into the pre-trained character detector or the pre-trained scene divider, the size of the water gauge image to be recognized is set, so that the calculation amount caused by overlarge image size can be reduced to a certain extent, and the calculation speed is increased.
As shown in fig. 5, as a possible implementation manner, in an embodiment of the method for automatically reading a water gauge scale, a training process of the character detector and the scene divider may specifically include:
s501, a training data set is obtained, wherein the training data set comprises a plurality of water gauge images and a first label and a second label of each water gauge image.
S502, a character detector is constructed, and the character detector is trained by using the water gauge image in the training data set and a first label of the water gauge image, wherein the character detector adopts a YOLOv5 target detection network, and the first label comprises numbers 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9 and letters E in the water gauge image.
Illustratively, the labelImg labeling software may be used to label the numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 and the letter E on the water gauge image, resulting in the first label.
Alternatively, the character detector may be pre-trained in the following manner, and the training process uses a YOLOv5 target detection network, and it should be understood that the network is composed of four parts, i.e., an input end, a feature extraction end, a feature fusion end, and an output end. The input end can be used for inputting a large amount of training data, the training data are expanded through the enhancement of the Mosaic data, the detection capability of small targets is improved, the data set is enriched, then the self-adaptive anchor frame calculation is carried out, and the optimal anchor frame suitable for the water gauge data set is calculated according to the size of the marking frame; the feature extraction end can be used for carrying out slicing operation on the input water gauge image to be recognized through the Focus structure so as to reduce the calculated amount and accelerate the training speed, and then the CSP structure is formed by connecting the CBL module, the Res unint module and the convolution layer, so that the learning capacity of the network can be increased, the light weight and the accuracy can be ensured; the feature fusion end adopts an FPN + PAN structure to perform multi-scale feature fusion, so that the diversity and robustness of features are further improved, and the predicted feature information is enhanced; the output end can be used for finishing the output of the character detection result on the water gauge, the GIOU _ Loss function is adopted for classifying and regressing the target frames, so that the network has better fitting, NMS non-maximum value inhibition is adopted in the post-processing stage, a plurality of target frames are screened, and the accuracy of network detection is improved. In practical application, the character detector trained by the method has training precision values of all categories over 93%, precision values over 95% and recall rate over 88%.
S503, constructing a scene divider, and training the scene divider by using the water gauge image in the training data set and a second label thereof, wherein the scene divider adopts a UNet semantic division network, and the second label comprises a water gauge region, a water surface region, a float grass region and a background region in the water gauge image.
For example, labelme labeling software can be used for labeling the water gauge region, the water surface region, the float grass region and the background region in the water gauge image to obtain the second label.
Optionally, the scene segmenter may be pre-trained in the following manner, and the training process adopts UNet semantic segmentation network, and it should be understood that the network is composed of a feature extraction end and a feature fusion end. In the feature extraction module, the input water gauge image to be identified is extracted into features of a deep layer and a shallow layer through a ResNet main network, all the main networks have four stages with residual blocks, and the stepping of the first layer convolution layer of each stage is 2, so that the obtained feature map is downsampled, the calculation efficiency is improved, and the receptive field is increased; in the feature fusion module, a PPM pyramid pooling module is adopted to capture context information, the obtained features are sent to an FPN feature pyramid module to be fused with deep features and shallow features, and a FAM stream alignment module is embedded in the FPN to replace normal bilinear interpolation to realize upsampling, so that abundant semantic representation of low-level features is provided for a network. In practical application, the scene divider trained by the method has the segmentation precision of each category of about 98%, the average merging ratio of 92% and the frequency-weight merging ratio of 96%.
According to the automatic reading method for the water gauge scales, before the water gauge image to be recognized is input into the pre-trained character detector or the pre-trained scene divider, the size of the water gauge image to be recognized is set, so that the calculated amount caused by overlarge image size can be reduced to a certain extent, and the calculating speed is increased; meanwhile, a deep learning algorithm is adopted to train the character detector and the scene divider, the detection capability of a small target is effectively improved, a more accurate relation between the water gauge and the water surface is obtained, and more accurate water gauge scales are obtained by analyzing the position relation between letters E around the numbers at the lowest part of the water gauge and the detected water surface. Meanwhile, the scale information of the water gauge is sent to the corresponding HTTP website and port number in a character string mode according to the water gauge reading, so that the client can call the water gauge reading at any time, and convenience is brought to further data analysis.
Furthermore, the embodiment of the invention also provides electronic equipment.
As shown in fig. 6, the electronic device 100 according to the embodiment of the present invention includes a memory 102, a processor 104, and a computer program 106 stored in the memory 102, wherein when the computer program 106 is executed by the processor 104, the automatic reading method for the water gauge scale according to the above-mentioned embodiment of the present invention is implemented.
Further, the embodiment of the invention also provides an automatic reading system for the water gauge scales.
As shown in fig. 7, the automatic water scale reading system 200 may include: a water gauge 201; a client 202, and a server 203. The client 202 is used for acquiring images of the water gauge 201 to obtain a water gauge image to be identified; and the server 203 is in communication connection with the client 202, and the electronic device 100 according to the above embodiment of the invention is adopted by the server 203.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (9)
1. A method for automatically reading scales on a water gauge is characterized by comprising the following steps:
acquiring a water gauge image to be identified;
inputting the water gauge image to be recognized into a pre-trained character detector to obtain a character detection frame, and inputting the water gauge image to be recognized into a pre-trained scene divider to obtain a scene division image;
obtaining the lowest number of the water gauge and the relation between the surrounding letter E of the lowest number and the water surface according to the character detection frame and the scene segmentation image, and obtaining the reading of the water gauge according to the lowest number and the relation;
the obtaining of the relationship between the lowest number of the water gauge and the surrounding letters E of the lowest number and the water surface according to the character detection frame and the scene segmentation image and the obtaining of the water gauge reading according to the lowest number and the relationship comprise:
sequencing the number detection frames in the character detection frames to obtain a number character string, matching the number character string with a preset number string, and obtaining the lowest number according to a matching result;
carrying out image fusion on the water surface area and the aquatic weed area in the scene segmentation image to obtain a water area;
judging whether the water gauge is in water or not according to the water area and a character detection frame corresponding to a peripheral letter E of the lowest number;
if so, obtaining the scale grade of a target letter E in water, and obtaining the water gauge reading according to the lowest number and the scale grade, wherein the target letter E is the letter E in the surrounding letters E in water;
the obtaining of the scale grade of the target letter E in water and the obtaining of the water gauge reading according to the lowest number and the scale grade comprises the following steps:
judging whether the right letter E of the lowest number is detected;
if yes, performing character grade speculation according to a detection frame corresponding to the right letter E of the lowest number;
when the right letter E of the lowest number is detected, determining the median of the heights of the detection frames corresponding to all the letters E on the water gauge, calculating the ratio of the heights of the detection frames corresponding to the detected right letter E of the lowest number to the median of the heights of the detection frames corresponding to all the letters E on the water gauge, determining the corresponding character grade according to the calculated ratio, and determining the height value of the detection frame corresponding to the right letter E of the corresponding lowest number according to the character grade;
determining a height value of the detection box corresponding to the lowest numerical right letter E by:
wherein, the first and the second end of the pipe are connected with each other,
the height value of the detection frame corresponding to the letter E on the right side of the lowest number is represented, and x represents the ratio of the height of the detection frame corresponding to the letter E on the right side of the lowest number to the median of the heights of the detection frames corresponding to all the letters E on the water gauge;
if the right letter E of the lowest number is not detected, calculating the distance between the lowest number and the water surface, and performing character grade estimation on the distance to obtain the height value of the detection frame corresponding to the right letter E of the lowest number;
judging whether the height value of a detection frame corresponding to the right letter E of the lowest number is smaller than a preset height or not;
if yes, determining the scale grade of the target letter E in the water as the grade of the letter E on the right side of the lowest number, and adding the height value of the detection frame corresponding to the letter E to the lowest number to obtain the final water gauge reading; otherwise, judging whether the lower letter E of the lowest number is detected;
if so, performing character grade speculation according to the detection frame corresponding to the lower letter E of the lowest number, obtaining a ratio, and judging whether the ratio is close to the height of the detection frame corresponding to the lower letter E of the lowest number;
if so, calculating the distance between the detection frame corresponding to the lower letter E of the lowest number and the water surface, and performing character grade estimation according to the distance and the ratio of the character detection frame, otherwise, performing character grade estimation according to the ratio obtained by performing character grade estimation on the detection frame corresponding to the lower letter E of the lowest number, and obtaining the height value of the detection frame corresponding to the lower letter E of the lowest number.
2. The automatic reading method of the water gauge scale according to claim 1, wherein the acquiring of the water gauge image to be identified comprises:
receiving byte information sent by a client through a target HTTP website and a port number, wherein the byte information is obtained by encoding the to-be-identified water gauge image by the client;
and decoding the byte information to obtain the water gauge image to be identified.
3. The method of automatic water scale reading according to claim 2, further comprising:
and sending the water gauge reading to the client through the target HTTP website and the port number.
4. The method of automatic reading of a water gauge scale of claim 1, further comprising:
before the character detection frames are sequenced, filtering out the character detection frames with the confidence degrees smaller than a preset confidence degree threshold value;
before determining a water area according to the scene segmentation image, setting a pixel point with a pixel value larger than a preset pixel value confidence coefficient threshold value in the scene segmentation image to be 255, and setting a pixel point with a pixel value smaller than or equal to the preset pixel value confidence coefficient threshold value to be 0.
5. The method of automatic water scale reading according to any one of claims 1-4, further comprising:
before the water gauge image to be recognized is input into a pre-trained character detector, setting the size of the water gauge image to be recognized as 640 x 640;
and before the water gauge image to be recognized is input into a pre-trained scene divider, setting the size of the water gauge image to be recognized as 512 x 512.
6. The method of claim 1, wherein the training process of the character detector and scene segmenter comprises:
acquiring a training data set, wherein the training data set comprises a plurality of water gauge images and a first label and a second label of each water gauge image;
constructing a character detector, and training the character detector by using a water gauge image in the training data set and a first label thereof, wherein the character detector adopts a YOLOv5 target detection network, and the first label comprises numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 and letters E in the water gauge image;
and constructing a scene divider, and training the scene divider by using the water gauge image in the training data set and a second label thereof, wherein the scene divider adopts a UNet semantic division network, and the second label comprises a water gauge area, a water surface area, a float grass area and a background area in the water gauge image.
7. The automatic water gauge scale reading method of claim 6,
labeling numbers 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9 and a letter E on the water gauge image by using labelImg labeling software to obtain a first label;
and labeling the water gauge region, the water surface region, the float grass region and the background region in the water gauge image by using labelme labeling software to obtain the second label.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory, wherein the computer program, when executed by the processor, implements the method of automatic water scale reading of any of claims 1-7.
9. A water gauge scale automatic reading system, the system comprising:
a water gauge;
the client is used for acquiring images of the water gauge to obtain a water gauge image to be identified;
a server communicatively connected to the client, the server using the electronic device according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211550233.XA CN115546793B (en) | 2022-12-05 | 2022-12-05 | Automatic reading method and system for water gauge scales and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211550233.XA CN115546793B (en) | 2022-12-05 | 2022-12-05 | Automatic reading method and system for water gauge scales and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115546793A CN115546793A (en) | 2022-12-30 |
| CN115546793B true CN115546793B (en) | 2023-04-18 |
Family
ID=84722342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211550233.XA Active CN115546793B (en) | 2022-12-05 | 2022-12-05 | Automatic reading method and system for water gauge scales and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115546793B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112598001A (en) * | 2021-03-08 | 2021-04-02 | 中航金城无人系统有限公司 | Automatic ship water gauge reading identification method based on multi-model fusion |
| WO2021238030A1 (en) * | 2020-05-26 | 2021-12-02 | 浙江大学 | Water level monitoring method for performing scale recognition on the basis of partitioning by clustering |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109522889A (en) * | 2018-09-03 | 2019-03-26 | 中国人民解放军国防科技大学 | Hydrological ruler water level identification and estimation method based on image analysis |
| CN110427933A (en) * | 2019-06-20 | 2019-11-08 | 浙江大学 | A kind of water gauge recognition methods based on deep learning |
| US20210374466A1 (en) * | 2020-05-26 | 2021-12-02 | Zhejiang University | Water level monitoring method based on cluster partition and scale recognition |
| CN112949624B (en) * | 2021-01-25 | 2024-02-13 | 西安电子科技大学 | Water gauge-based water level detection method and device, electronic equipment and storage medium |
| CN113971779B (en) * | 2021-10-29 | 2022-07-01 | 中国水利水电科学研究院 | Water gauge automatic reading method based on deep learning |
-
2022
- 2022-12-05 CN CN202211550233.XA patent/CN115546793B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021238030A1 (en) * | 2020-05-26 | 2021-12-02 | 浙江大学 | Water level monitoring method for performing scale recognition on the basis of partitioning by clustering |
| CN112598001A (en) * | 2021-03-08 | 2021-04-02 | 中航金城无人系统有限公司 | Automatic ship water gauge reading identification method based on multi-model fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115546793A (en) | 2022-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506798B (en) | Water level monitoring method based on image recognition | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| CN111723748A (en) | Infrared remote sensing image ship detection method | |
| CN111275040B (en) | Positioning method and device, electronic equipment and computer readable storage medium | |
| CN115333678B (en) | Unmanned ship water quality monitoring data transmission method and system | |
| CN110909640A (en) | Method and device for determining water level line, storage medium and electronic device | |
| CN113065578A (en) | Image visual semantic segmentation method based on double-path region attention coding and decoding | |
| CN114511718B (en) | Intelligent management method and system for materials for building construction | |
| CN109800756A (en) | A kind of text detection recognition methods for the intensive text of Chinese historical document | |
| CN115359366A (en) | Remote sensing image target detection method based on parameter optimization | |
| CN115880571A (en) | Water level gauge reading identification method based on semantic segmentation | |
| CN110349134B (en) | Pipeline disease image classification method based on multi-label convolutional neural network | |
| CN116862910B (en) | Visual detection method based on automatic cutting production | |
| CN115546793B (en) | Automatic reading method and system for water gauge scales and electronic equipment | |
| CN110837834B (en) | Digital instrument reading method and system | |
| CN112991342B (en) | Water level line detection method, device and system based on water level gauge image | |
| CN114419443A (en) | Automatic remote-sensing image cultivated land block extraction method and system | |
| CN115081341B (en) | Basin flood simulation early warning method and system | |
| CN113177513B (en) | Method, device, equipment and storage medium for detecting wearing of safety helmet | |
| CN117079197B (en) | Intelligent building site management method and system | |
| CN117372871A (en) | Building extraction method based on remote sensing image | |
| CN116229475A (en) | Document identification method and device and electronic equipment | |
| CN113537197A (en) | Meter automatic modeling method based on machine vision | |
| CN117789200A (en) | Fruit point cloud extraction method and device, electronic equipment and storage medium | |
| CN117934607A (en) | Pedestrian target detection method, device, equipment and medium based on monitoring scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |