CN117036993A - Ship water gauge remote measurement method based on unmanned aerial vehicle - Google Patents
Ship water gauge remote measurement method based on unmanned aerial vehicle Download PDFInfo
- Publication number
- CN117036993A CN117036993A CN202310821432.8A CN202310821432A CN117036993A CN 117036993 A CN117036993 A CN 117036993A CN 202310821432 A CN202310821432 A CN 202310821432A CN 117036993 A CN117036993 A CN 117036993A
- Authority
- CN
- China
- Prior art keywords
- water gauge
- water
- water surface
- gauge
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 284
- 238000000691 measurement method Methods 0.000 title claims abstract description 18
- 238000001514 detection method Methods 0.000 claims abstract description 49
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 43
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000011218 segmentation Effects 0.000 claims abstract description 30
- 230000006870 function Effects 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 5
- 238000003064 k means clustering Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 4
- 238000012805 post-processing Methods 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 2
- 238000005260 corrosion Methods 0.000 claims description 2
- 238000003708 edge detection Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 abstract description 20
- 238000013135 deep learning Methods 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000002474 experimental method Methods 0.000 abstract description 2
- 238000004364 calculation method Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005303 weighing Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013215 result calculation Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/1444—Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Probability & Statistics with Applications (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a ship water gauge remote measurement method based on an unmanned aerial vehicle, which is characterized in that a water gauge draft value is detected by adopting a water gauge measurement system mainly composed of a WEB main control end, a terminal, an algorithm end and a database, an original video containing a water surface area and a water gauge is collected by the unmanned aerial vehicle, then the original video is screened at the terminal to obtain an effective picture, and then the algorithm end analyzes the effective picture by an image recognition method to obtain the water gauge draft value; the WEB main control terminal manages the user, the data and the authority, issues a task instruction to the terminal and receives an operation result of the algorithm terminal. According to the invention, by utilizing the advantages of high efficiency and flexibility of the unmanned aerial vehicle, combining the technologies of deep learning target detection, character recognition, target segmentation and the like, optimizing and improving, the defects of safety and convenience of water gauge measurement operation can be overcome, and a more accurate measurement result can be obtained; experiments prove that the invention has the advantages of high running speed, high detection precision and flexible detection area.
Description
Technical Field
The invention relates to a ship water gauge remote measurement method based on an unmanned aerial vehicle, which is a ship water gauge scale detection technology.
Background
The weighing of the ship water gauge is the most one metering mode used in the bulk cargo transportation industry of ships at home and abroad at present, and the reading of the draft value of the six water gauges of the ship becomes the most critical factor of the weighing accuracy of the ship water gauge. Because of the lack of a special ship water gauge weighing measuring instrument, the draft value is mainly read by adopting a human eye observation mode at present, and the mode has the problems of subjective factors, incapability of recording data in the observation process, few data of observation points and the like, which can lead to inaccurate and inaccurate readings; in addition, because the observation environment is inconvenient, the safety risk of personnel is increased, the workload is high, and the labor cost is high.
In addition to the conventional visual inspection method for eyes, various schemes are attempted to solve the problem at present, such as detecting the draft of a ship side by installing a double pressure sensor, transmitting ultrasonic waves through an ultrasonic sensor and calculating position information of a waterline according to the time required for reflecting the ultrasonic waves from the water surface back to a deck, directly obtaining the water surface position by using a laser ranging sensor, and the like, and the methods are all based on physical sensors, and all have the problems of difficult installation, difficult construction, easy environmental influence of precision, and the like.
There are also some methods for automatically reading water gauge based on image video algorithm, and the image recognition method mainly comprises a waterline positioning module, a character recognition module and a reading numerical value estimating module. In the traditional image recognition method, the position of the water gauge line is detected through the edge of image processing, the digital position of the scale is positioned through image binarization, recognition is carried out through methods such as template matching, and finally the draft scale value of the water gauge is estimated through fitting; the method is poor in generalization and robustness, is limited to partial water gauge scenes, is easily influenced by factors such as ship rust, water line and the like, and cannot guarantee water line detection positioning and digital identification precision.
Related methods based on deep learning are proposed in recent years, the methods realize scale character recognition and positioning by using a training deep learning network, position waterline position information by using an image segmentation network, and the two methods are combined to estimate the waterline by using methods such as a least square method and the like; the problem of the method is that the error recognition rate of character positioning and recognition is high for the whole image, and the bounding boxes positioned by the characters are not attached enough, so that larger reading errors can be caused; in addition, the method can not well solve the problem that the water gauge has inclination and rotation angle in the image; and because the ship body has a rotation inclination angle in a non-plane, the numerical value of the scale character and the interval are in a non-linear state, and the reading error is easy to increase.
Disclosure of Invention
The invention aims to: in order to overcome the defects in the prior art, the invention provides a ship water gauge remote measurement method based on an unmanned aerial vehicle, so as to improve the accuracy of port water gauge reading and improve the working efficiency.
The technical scheme is as follows: in order to achieve the above purpose, the invention adopts the following technical scheme:
a ship water gauge remote measurement method based on an unmanned aerial vehicle is characterized in that a water gauge draft value is detected by adopting a water gauge measurement system mainly composed of a WEB main control end, a terminal, an algorithm end and a database, an original video containing a water surface area and a water gauge is collected by the unmanned aerial vehicle, then the original video is screened at the terminal to obtain an effective picture, and then the algorithm end analyzes the effective picture through an image recognition method to obtain the water gauge draft value; the WEB main control terminal manages the user, the data and the authority, issues a task instruction to the terminal and receives an operation result of the algorithm terminal.
In the water gauge measuring system adopted by the invention, the WEB main control end is generally used by an administrator, and the terminal is generally a mobile terminal (such as a mobile phone or a notebook computer) and is used by field operators. For one specific draft value measurement, the flow is approximately as follows: firstly, an administrator creates a task at a WEB main control end and issues the task to a field operator; secondly, on-site operators check the work tasks belonging to the operators on the mobile terminal, click corresponding functions, mobilize the unmanned aerial vehicle to acquire video of the ship water gauge, and upload the video to the server; then, the algorithm end carries out artificial intelligence based identification on the received original video to obtain a draft value result of the water gauge, and synchronizes the result to the WEB main control end and the mobile terminal; the field operator can correct the feedback result on site, and the corrected result is sent to the WEB main control end for reference; and finally, uniformly writing data such as the measurement result, the correction result and the like of the original video, the picture and the draft value of the water gauge into a database by the WEB main control end to finish the task.
Preferably, the WEB main control end, the terminal, the algorithm end and the database are completely decoupled, and are deployed in a restful interface mode so as to facilitate subsequent function expansion and upgrading.
Preferably, the original video is screened to obtain an effective picture, the effective picture meets the resolution requirement (the picture cannot be too fuzzy and the character area cannot be too small) and simultaneously comprises a water surface area and a water gauge, wherein the water surface traverses the whole effective picture and the area below the water surface accounts for 1/3+/-alpha of the area of the whole effective picture, the water gauge stands on the vertical midline of the whole effective picture, the range of the water gauge which deviates from the vertical midline transversely is limited within 10% of the width of the whole effective picture, the angle of the water gauge which deviates from the vertical midline is limited within +/-30 DEG, at least one letter M appears in the water gauge, and the exposed length of the water gauge in the whole picture is 2-2.5M (namely, in the whole picture, only a small water gauge in the range of 2-2.5M can be seen, for example, the water gauge in the range of 5-7M or 1-3.5M is exposed in the whole picture); alpha is a set value; and setting the gesture of the unmanned aerial vehicle during acquisition based on the requirement on the effective picture.
In the prior art, the water gauge measurement mainly relies on manual work to accomplish the collection of original video, and the mode, angle, the definition etc. that the video was shot do not have unified standard, and the original video of gathering back can't realize effective, canonical structuring and keep. The invention has the important design point that the unmanned aerial vehicle is used in the acquisition of the original video, so that the more severe shooting requirement can be provided, the later data processing is convenient, and the detection accuracy can be improved. The requirement of the effective picture in the later period is combined, the shooting requirement of the unmanned aerial vehicle can be standardized, the shooting requirement of the unmanned aerial vehicle is limited to be the requirement of the effective picture or slightly relaxed, the original video shot according to the requirement is suitable, the accuracy of automatic identification of an algorithm is greatly improved, and the real reading of the water gauge can be clearly and clearly seen by on-site personnel.
Preferably, the algorithm end comprises a water gauge area detection module, a water gauge character positioning module, a character recognition module, a water gauge fitting module, a water surface detection module, a water gauge reading module and a water gauge reading verification module;
the water gauge area detection module is used for picking out a water gauge area from the whole effective picture;
the water gauge character positioning module is used for positioning coordinates and boundaries of each character in a water gauge area;
the character recognition module recognizes characters in the water gauge area;
the water gauge fitting module is used for reconstructing a complete water gauge by combining the coordinates and the recognition result of each character;
the water surface detection module is used for detecting the boundary between the water surface and the hull in the whole effective picture to obtain a water surface boundary point;
the water gauge reading module is used for reading the water gauge value by combining the reconstructed complete water gauge and the water surface junction point;
and the water gauge reading verification module is used for combining all the read water gauge values, and removing error values through a fitting function to obtain optimized water gauge values.
Preferably, the water gauge area detection module adopts a YOLO algorithm to detect the target.
Preferably, after the water gauge area is extracted, the water gauge area detection module performs image processing including edge detection, binarization and expansion corrosion on the water gauge area to obtain an external rectangle of the character area, and calculates a rotation translation matrix for correcting the inclination of the water gauge area according to the inclination angle and the rotation center of the external rectangle of the character area; the water gauge area is subjected to inclination correction according to the rotation translation matrix and then provided for a water gauge character positioning module and a character recognition module; and correcting the inclination of the whole effective picture according to the rotation translation matrix, and providing the corrected inclination to a water surface detection module.
Preferably, the water gauge character positioning module adopts a text segmentation method based on image segmentation, namely, classification is carried out from a pixel layer, text targets of each pixel point are judged, then all the text targets are integrated to obtain a probability map of a text segmentation area, and finally, a surrounding curve of the text segmentation area is obtained in a post-processing mode.
The accuracy of the current artificial intelligence algorithm cannot reach complete accuracy, so that the identification result is always unreliable, and a directly generated water gauge result has a larger error. Furthermore, in a dock, port environment, the water surface is not completely stationary, but exhibits irregular fluctuations. Therefore, the identified water gauge needs to be reconstructed, and standardized fitting is performed on the horizontal plane at the same time, so that accuracy of water gauge reading is improved. Preferably, when the water gauge fitting module performs water gauge reconstruction, a plurality of priori standards can be fused in a water gauge reconstruction algorithm, so that the reconstructed water gauge is more standard and accurate; these a priori criteria are for example: 1. there should be and only one "8632" between two "M" characters; 2. '8622' is in a vertical column, and the character spacing is not very different; 3. before each character "M", there should be 1-bit or 2-bit characters; 4. if there is a two digits before the character "M", the first digit should be no more than 3; 5. all characters should be relatively aggregated in the transverse direction and relatively discrete in the vertical direction.
Preferably, the water surface detection module determines a boundary between the water surface and the hull by adopting a two-stage image segmentation method, performs primary segmentation on the water surface position of the whole effective picture during the first-stage image segmentation, and takes a partial picture (such as a partial picture with the size of 512×512) upwards from the water surface obtained by the primary segmentation or takes a partial picture (such as a partial picture with the size of 512×512) downwards from the nearest complete character of the water surface obtained by the primary segmentation during the second-stage image segmentation, and performs secondary segmentation on the water surface position of the partial picture, and takes the result of the secondary segmentation as the boundary between the water surface and the hull. The two picture segmentation methods may be the same or different, such as both employing the envelope algorithm mentioned herein.
Considering that the waterline (water surface) is formed by water waves, the influence of the fluctuation of water is finally calculated, the calculated value at the wave crest is higher than the true value, and the calculated value at the wave trough is lower than the true value, so that a water wave curve needs to be processed to obtain a more accurate waterline. Preferably, the water surface detection module calculates the water surface position by adopting an envelope curve algorithm, and specifically comprises:
the following envelope:
upper envelope:
water surface position:
wherein: y (t) represents the calculated value of the water surface position at the time t, y down (t) represents the position of the water surface after correction at t on the lower envelope, y up (t) represents the position of the water surface at time t on the corrected upper envelope; y is 0 (t) represents the actual position of the water surface at time t, down (t) represents the fitting value of the water surface position at time t obtained by the lower envelope fitting function, and up (t) represents the fitting value of the water surface position at time t obtained by the upper envelope fitting function.
A section of original video can read out a plurality of groups of waterline results, and some of the waterline results have larger errors with the true value and some of the waterline results have smaller errors due to errors or error problems of recognition or calculation; in general, the larger-deviation values account for a smaller proportion of the overall, and common errors are similar, so correct or erroneous data will typically be concentrated around several values. Preferably, the water surface detection module performs unsupervised learning on the obtained values of all water surface positions by adopting a K-means clustering method (also called as a K-means algorithm), finds a clustering center of the values, deletes all values of clusters with fewer values, circulates in this way until a cost function of the K-means clustering method is met, reserves the values around the final clustering center, and takes the values of the finally reserved water surface positions as water surface boundary points for reading the values of the water gauge.
The algorithm is adopted to read the draft value of the water gauge, so that the requirement on hardware resources is high, the time consumption is high, and the logic relationship is complex; therefore, the performance of the server can be considered, and the six-surface video of the ship is operated in a multipath parallel mode, so that the time consumption of the system is reduced by 6 times. Preferably, all algorithms in the water gauge measurement system are packaged by adopting standard interfaces, so that a code enabling effect is achieved, and later maintenance and upgrading are facilitated; to six water gauge draft values that same bow boats and ships need to be measured, adopt six unmanned aerial vehicle simultaneous working's mode to carry out original video acquisition to six of boats and ships, water gauge measurement system adopts the parallel mode of multichannel to calculate six original videos simultaneously, obtains six water gauge draft values in step.
The beneficial effects are that: according to the unmanned aerial vehicle-based ship water gauge remote measurement method, the advantages of high efficiency and flexibility of the unmanned aerial vehicle are utilized, technologies such as deep learning target detection, character recognition and target segmentation are combined, optimization and improvement are carried out, the defects of safety and convenience of water gauge measurement operation can be overcome, a more accurate measurement result is obtained, and on-site staff can safely and conveniently finish measurement tasks through a handheld terminal. Experiments prove that the invention has the advantages of high running speed, high detection precision, strong timeliness, flexible detection area and reliable water gauge acquisition and measurement result, can lighten the working pressure of field operators, reduce the operation cost, improve the overall operation efficiency and the operation safety, can avoid errors caused by personal subjective factors, and further can avoid the occurrence of three-party disputes.
Drawings
FIG. 1 is a block diagram of a water gauge measurement system of the present invention;
FIG. 2 is a flow chart of the algorithm end of the invention for reading the water gauge value;
fig. 3 is an example of a inland port identification result in the embodiment;
fig. 4 is an example of identification results of coastal ports in the embodiment.
Detailed Description
The invention is described in detail below with reference to the drawings and the specific embodiments.
A ship water gauge remote measurement method based on an unmanned aerial vehicle is characterized in that a water gauge draft value is detected by a water gauge measurement system mainly composed of a WEB main control end, a handheld terminal, an algorithm end and a database, an original video containing a water surface area and the water gauge is collected through the unmanned aerial vehicle, then the original video is screened at the handheld terminal to obtain an effective picture, and then the algorithm end analyzes the effective picture through an image recognition method to obtain the water gauge draft value. The WEB main control terminal is mainly responsible for data management, result inquiry, user management, authority management, task management, system scheduling and the like; the handheld terminal is mainly responsible for data acquisition, data processing, data transmission, starting identification and the like; the algorithm end is mainly responsible for carrying out data analysis, intelligent identification, result calculation and the like on the collected original video; the database is mainly responsible for the structured storage of all data. And the WEB main control end, the terminal, the algorithm end and the database are completely decoupled, and are deployed in a restful interface mode so as to facilitate subsequent function expansion and upgrading.
In this example, the water gauge measurement system adopts a scheme of a WEB master control end and an operator handheld terminal deployed by private cloud, and the architecture is shown in fig. 1. The WEB main control end serves as a main control center of the whole water gauge measuring system, a task is issued to the handheld terminal, an operator controls the unmanned aerial vehicle to complete shooting of an original video, the original video is imported into the handheld terminal in a wireless transmission or manual import mode, screening of effective pictures is completed through the handheld terminal, the screened effective pictures are sent to the algorithm end to conduct AI core calculation, and a calculation result is sent to an existing middle inspection platform to generate a final inspection report. The AI core calculation and the data storage and management are all executed in the cloud server. The main parameters of the same system are shown in table 1:
TABLE 1 System principal parameters
In the embodiment, one inland port and one coastal port are selected, the ship water gauge is measured remotely, and the ship water gauge is compared with the manual reading, so that the feasibility of the system is verified.
The unmanned aerial vehicle in this example refers to the requirement setting shooting requirement of the effective picture, specifically: the video frame meets the resolution requirement (the picture cannot be too fuzzy, the character area cannot be too small) and simultaneously comprises a water surface area and a water gauge, wherein the water surface traverses the whole video frame, the area below the water surface occupies 1/3+/-alpha of the area of the video frame, the water gauge stands on the vertical central line of the video frame, the range of the water gauge which deviates from the vertical central line transversely is limited within 10% of the width of the video frame, the angle of the water gauge which deviates from the vertical central line is limited within +/-30 degrees, at least one letter M appears in the water gauge, and the length of the water gauge which is exposed in the whole picture is 2-2.5M.
The algorithm end of the embodiment mainly comprises a water gauge area detection module, a water gauge character positioning module, a character recognition module, a water gauge fitting module, a water surface detection module, a water gauge reading module and a water gauge reading verification module; the water gauge area detection module is used for picking out a water gauge area from the whole effective picture; the water gauge character positioning module is used for positioning coordinates and boundaries of each character in a water gauge area; the character recognition module recognizes characters in the water gauge area; the water gauge fitting module is used for reconstructing a complete water gauge by combining the coordinates and the recognition result of each character; the water surface detection module is used for detecting the boundary between the water surface and the hull in the whole effective picture to obtain a water surface boundary point; the water gauge reading module is used for reading the water gauge value by combining the fitted complete water gauge and the water surface junction; and the water gauge reading verification module is used for combining all the read water gauge values, and removing error values through a fitting function to obtain optimized water gauge values.
As shown in fig. 3, the method is a recognition result of a ship water gauge at a inland port, and is mainly characterized in that: the water surface is relatively calm, the wind and wave are small, and the influence of scale recognition fluctuation range is small; the ship is less in type and relatively fixed in characteristics; there are many nonstandard ships, and it is difficult to solve all cases.
As shown in fig. 4, the identifying result of the water gauge of the coastal port ship is mainly characterized in that: the water surface is relatively fluctuant, the wind wave is larger, and the influence of scale recognition fluctuation range is larger; the types of ships are more, and more data need to be collected for learning and training; fewer nonstandard ships exist, and most of ships can be successfully identified.
After the field operator finishes the operation according to the system flow, the collected original video, the water gauge identification result video, the water gauge representative frame image, the water gauge result set and the task related information (such as port/wharf information, operator information, unmanned aerial vehicle information and the like) are stored in the database in a structured mode, so that the follow-up tracing and searching are facilitated.
The water gauge area detection module in this example aims at judging the position of the water gauge from the input picture by adopting a target detection algorithm, and is a very core task. In the project investigation process, the method evaluates various target detection algorithms, and finds that: under the same speed standard, the YOLO algorithm is the most accurate; under the same precision standard, the YOLO algorithm is fastest. Thus, the example selects the YOLO algorithm as the target detection algorithm.
The character positioning module of the water gauge in the embodiment aims at positioning the position of characters from the detected water gauge area by adopting a character detection algorithm, wherein the character detection algorithm adopts a text segmentation method based on image segmentation, namely, classification is carried out from a pixel layer, text targets to which each pixel point belongs are judged, then all the text targets are integrated to obtain a probability map of the text segmentation area, and finally, an enclosing curve of the text segmentation area is obtained in a post-processing mode.
The character recognition module in this example aims to recognize characters in a region after a character detection algorithm is adopted to locate the character region in a picture. In popular terms, the character detection task solves the problem of where the characters are in the image, and the character recognition task solves the problem of what the characters are in the image.
The water gauge fitting module in this example is mainly to the water gauge that discerns reconfigurations to improve the accuracy of water gauge. In the water gauge reconfiguration process, the priori standard is adopted to correct the recognized characters, and accuracy of the reconfiguration water gauge is improved.
The water surface detection module in the embodiment aims to realize the positions of the water surface and the hull by adopting a segmentation method and find the position of the waterline. The segmentation method comprises semantic segmentation and instance segmentation: semantic segmentation is an expansion of foreground and background separation, requiring separation of image parts with different semantics; while the expansion of the detection task during the instance segmentation requires the profile of the object at the description (finer than the detection box).
And carrying out unsupervised learning on the values of all the water surface positions determined by the water surface detection module, finding out a clustering center of the values, deleting all the values of the clusters with fewer values, circulating until the value function of the K-means clustering method is met, reserving the values around the final clustering center, and taking the finally reserved values of the water surface positions as water surface boundary points for reading the values of the water gauge.
The water gauge reading module in this case combines the complete water gauge of reconstruction and surface of water juncture, reads the water gauge numerical value.
And the water gauge reading verification module in the embodiment is used for combining all the read water gauge values, and eliminating error values through a fitting function to obtain optimized water gauge values.
In general, at the algorithm end, the accuracy of the existing method for reading the water gauge value is optimized by the methods of amplifying the data set, adding a water gauge fitting module according to the priori condition, adopting a local water surface segmentation method, screening the value of the water surface position through a K-means clustering method and the like.
TABLE 2 comparative test results using the prior art method and the inventive method
As can be seen from Table 2, the detection precision and the detection efficiency of the method are obviously improved, the photographed original video is clear and standard, and under the condition that the waterline detection, character frame selection and recognition accuracy are higher, the error of the final reading result is obviously reduced.
In addition, the system device of the present invention can support two configurations:
1. manual version: the unmanned plane, the tablet personal computer and the server realize automatic identification and are required to be controlled by a person;
2. automatic version: unmanned aerial vehicle + unmanned aerial vehicle nest, autonomous flight, automatic identification realizes real unmanned.
Meanwhile, the system can be popularized to various domestic wharfs and ports; the method can be widely applied to scenes such as water level measurement, water depth measurement, oil level measurement and object volume measurement, can be popularized to various fields such as water conservancy, energy, production and manufacturing, teaching and the like, can save a large amount of manpower and material resources, and has a huge future market prospect.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be appreciated by persons skilled in the art that the above embodiments are not intended to limit the invention in any way, and that all technical solutions obtained by means of equivalent substitutions or equivalent transformations fall within the scope of the invention.
Claims (10)
1. A ship water gauge remote measurement method based on unmanned aerial vehicle is characterized in that: detecting the draft value of the water gauge by adopting a water gauge measuring system mainly composed of a WEB main control end, a terminal, an algorithm end and a database, firstly acquiring an original video containing a water surface area and the water gauge by an unmanned aerial vehicle, then screening the original video at the terminal to obtain an effective picture, and then analyzing the effective picture by the algorithm end through an image recognition method to obtain the draft value of the water gauge; the WEB main control terminal manages the user, the data and the authority, issues a task instruction to the terminal and receives an operation result of the algorithm terminal.
2. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 1, wherein: screening an original video to obtain an effective picture, wherein the effective picture meets resolution requirements and simultaneously comprises a water surface area and a water gauge, the water surface traverses the whole effective picture, the area below the water surface accounts for 1/3+/-alpha of the area of the whole effective picture, the water gauge stands on the vertical central line of the whole effective picture, the range of the water gauge, which deviates from the vertical central line transversely, is limited within 10% of the width of the whole effective picture, the angle of the water gauge, which deviates from the vertical central line, is limited within +/-30 ℃, at least one letter M appears in the water gauge, and the exposed length of the water gauge in the whole picture is 2-2.5M; alpha is a set value; and setting the gesture of the unmanned aerial vehicle during acquisition based on the requirement on the effective picture.
3. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 1, wherein: the algorithm end comprises a water gauge area detection module, a water gauge character positioning module, a character recognition module, a water gauge fitting module, a water surface detection module, a water gauge reading module and a water gauge reading verification module;
the water gauge area detection module is used for picking out a water gauge area from the whole effective picture;
the water gauge character positioning module is used for positioning coordinates and boundaries of each character in a water gauge area;
the character recognition module recognizes characters in the water gauge area;
the water gauge fitting module is used for reconstructing a complete water gauge by combining the coordinates and the recognition result of each character;
the water surface detection module is used for detecting the boundary between the water surface and the hull in the whole effective picture to obtain a water surface boundary point;
the water gauge reading module is used for reading the water gauge value by combining the reconstructed complete water gauge and the water surface junction point;
and the water gauge reading verification module is used for combining all the read water gauge values, and removing error values through a fitting function to obtain optimized water gauge values.
4. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3, wherein: the water surface detection module adopts a two-stage image segmentation method to determine the boundary between the water surface and the hull, performs primary segmentation on the water surface position of the whole effective picture during primary image segmentation, and takes a local sample upwards from the water surface obtained by primary segmentation or takes a local sample downwards from the nearest complete character of the water surface obtained by primary segmentation during secondary segmentation, and performs secondary segmentation on the water surface position of the local sample, wherein the result of the secondary segmentation is taken as the boundary between the water surface and the hull.
5. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3 or 4, wherein: the water surface detection module calculates the water surface position by adopting an envelope curve algorithm, and specifically comprises the following steps:
the following envelope:
upper envelope:
water surface position:
wherein: y (t) represents the calculated value of the water surface position at the time t, y down (t) represents the position of the water surface after correction at t on the lower envelope, y up (t) represents the position of the water surface at time t on the corrected upper envelope; y is 0 (t) represents the actual position of the water surface at time t, and Down (t) represents the water surface position at time t obtained by the lower envelope fitting functionFitting values, up (t), represent fitting values of the water surface position at time t obtained by the upper envelope fitting function.
6. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3 or 4, wherein: and the water surface detection module performs unsupervised learning on the obtained values of all the water surface positions by adopting a K-means clustering method, and takes the finally-reserved values of the water surface positions as water surface boundary points for reading the values of the water gauge.
7. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3, wherein: after the water gauge area detection module is used for picking up the water gauge area, image processing including edge detection, binarization and expansion corrosion is adopted for the water gauge area, the external rectangle of the character area is obtained, and a rotation translation matrix for correcting the inclination of the water gauge area is calculated according to the inclination angle and the rotation center of the external rectangle of the character area; the water gauge area is subjected to inclination correction according to the rotation translation matrix and then provided for a water gauge character positioning module and a character recognition module; and correcting the inclination of the whole effective picture according to the rotation translation matrix, and providing the corrected inclination to a water surface detection module.
8. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3, wherein: and the water gauge area detection module is used for carrying out target detection by adopting a YOLO algorithm.
9. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3, wherein: the water gauge character positioning module adopts a text segmentation method based on image segmentation, namely, classification is carried out from a pixel layer, text targets of each pixel point are judged, then all the text targets are integrated to obtain a probability map of a text segmentation area, and finally, a surrounding curve of the text segmentation area is obtained in a post-processing mode.
10. The unmanned aerial vehicle-based ship water gauge remote measurement method according to claim 3, wherein: the water gauge fitting module fuses the priori standard of the water gauge into the water gauge reconstruction algorithm, corrects the coordinates and the recognition result of each character, and reconstructs the complete water gauge.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310821432.8A CN117036993A (en) | 2023-07-06 | 2023-07-06 | Ship water gauge remote measurement method based on unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310821432.8A CN117036993A (en) | 2023-07-06 | 2023-07-06 | Ship water gauge remote measurement method based on unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036993A true CN117036993A (en) | 2023-11-10 |
Family
ID=88641942
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310821432.8A Pending CN117036993A (en) | 2023-07-06 | 2023-07-06 | Ship water gauge remote measurement method based on unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036993A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117705243A (en) * | 2024-02-06 | 2024-03-15 | 中理检验有限公司 | Water gauge weighing method and system based on mobile ultrasonic sensor |
| CN117788463A (en) * | 2024-02-26 | 2024-03-29 | 中邮建技术有限公司 | Ship draft detection method based on video AI and multi-mode data fusion |
-
2023
- 2023-07-06 CN CN202310821432.8A patent/CN117036993A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117705243A (en) * | 2024-02-06 | 2024-03-15 | 中理检验有限公司 | Water gauge weighing method and system based on mobile ultrasonic sensor |
| CN117705243B (en) * | 2024-02-06 | 2024-04-30 | 中理检验有限公司 | Water gauge weighing method and system based on mobile ultrasonic sensor |
| CN117788463A (en) * | 2024-02-26 | 2024-03-29 | 中邮建技术有限公司 | Ship draft detection method based on video AI and multi-mode data fusion |
| CN117788463B (en) * | 2024-02-26 | 2024-05-10 | 中邮建技术有限公司 | Ship draft detection method based on video AI and multi-mode data fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113705478B (en) | Mangrove single wood target detection method based on improved YOLOv5 | |
| CN117036993A (en) | Ship water gauge remote measurement method based on unmanned aerial vehicle | |
| CN110889324A (en) | Thermal infrared image target identification method based on YOLO V3 terminal-oriented guidance | |
| CN111091095B (en) | Method for detecting ship target in remote sensing image | |
| CN111368690B (en) | Deep learning-based video image ship detection method and system under influence of sea waves | |
| CN116148808B (en) | Automatic driving laser repositioning method and system based on point cloud descriptor | |
| CN109711353B (en) | Ship waterline area identification method based on machine vision | |
| CN102975826A (en) | Portable ship water gauge automatic detection and identification method based on machine vision | |
| CN110619328A (en) | Intelligent ship water gauge reading identification method based on image processing and deep learning | |
| CN114415168A (en) | Unmanned surface vessel track fusion method and device | |
| CN114067118B (en) | Processing method of aerial photogrammetry data | |
| CN108921165A (en) | Water level recognition methods based on water gauge image | |
| CN112418028A (en) | Satellite image ship identification and segmentation method based on deep learning | |
| CN116824570B (en) | Draught detection method based on deep learning | |
| CN116935369A (en) | Ship water gauge reading method and system based on computer vision | |
| CN114549909A (en) | Pseudo label remote sensing image scene classification method based on self-adaptive threshold | |
| CN114926400A (en) | Fan blade defect detection method based on improved YOLOv5 | |
| CN114155411A (en) | Intelligent detection and identification method for small and weak targets | |
| CN113344148A (en) | Marine ship target identification method based on deep learning | |
| CN117593601A (en) | Water gauge tide checking method based on deep learning | |
| CN116486212A (en) | Water gauge identification method, system and storage medium based on computer vision | |
| CN116051885A (en) | Processing and mesoscale vortex identification method for marine mesoscale vortex sample data | |
| CN116363655A (en) | Financial bill identification method and system | |
| CN114463628A (en) | Deep learning remote sensing image ship target identification method based on threshold value constraint | |
| CN116385915A (en) | Water surface floater target detection and tracking method based on space-time information fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |