CN114973261B - Method for calculating operation amount of water surface cleaning ship - Google Patents

Method for calculating operation amount of water surface cleaning ship Download PDF

Info

Publication number
CN114973261B
CN114973261B CN202210921088.5A CN202210921088A CN114973261B CN 114973261 B CN114973261 B CN 114973261B CN 202210921088 A CN202210921088 A CN 202210921088A CN 114973261 B CN114973261 B CN 114973261B
Authority
CN
China
Prior art keywords
image
garbage
scale
calculating
garbage bin
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210921088.5A
Other languages
Chinese (zh)
Other versions
CN114973261A (en
Inventor
余道洋
李梁
盛林华
陆炜
朱丽燕
苏厉
许峰
朱靖
姚欣欣
秦晟铭
刘锦淮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Institutes of Physical Science of CAS
Original Assignee
Hefei Institutes of Physical Science of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Institutes of Physical Science of CAS filed Critical Hefei Institutes of Physical Science of CAS
Priority to CN202210921088.5A priority Critical patent/CN114973261B/en
Publication of CN114973261A publication Critical patent/CN114973261A/en
Application granted granted Critical
Publication of CN114973261B publication Critical patent/CN114973261B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G06V30/147—Determination of region of interest
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01F—MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
    • G01F23/00—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm
    • G01F23/04—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm by dip members, e.g. dip-sticks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/164—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/1801—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/02—Recognising information on displays, dials, clocks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A20/00—Water conservation; Efficient water supply; Efficient water use
    • Y02A20/20—Controlling water pollution; Waste water treatment
    • Y02A20/204—Keeping clear the surface of open water from oil spills

Abstract

The invention discloses a method for calculating the operation quantity of a water surface cleaning ship, which comprises an internal standard ruler arranged in a garbage bin of the water surface cleaning ship and an image acquisition device for acquiring a real-time image of the ruler, wherein the method for calculating the operation quantity of the water surface cleaning ship comprises the following steps: step 1, acquiring a real-time image of a scale through an image acquisition device; step 2, calculating the real-time height of the garbage in the garbage bin according to the real-time image obtained in the step 1; and 3, calculating the amount of the garbage stored in the garbage bin according to the real-time height of the garbage in the garbage bin and the size of the garbage bin acquired in the step 2.

Description

Method for calculating operation amount of water surface cleaning ship
Technical Field
The invention relates to the technical field of cleaning ships, in particular to a method for calculating the operation quantity of a water surface cleaning ship.
Background
A water surface cleaning ship is equipment for salvaging water surface garbage, and mainly has the functions of salvaging, collecting and storing the water surface garbage in a target water area along the course of the ship, and then transporting the water surface garbage to a designated position for treatment.
The existing water surface cleaning ship is provided with a garbage collection cabin for storing garbage fished by the cleaning ship from the water surface, after the water surface cleaning ship runs for a period of time, continuous manual inspection is needed, the capacity of the garbage accumulated in the garbage collection cabin of the cleaning ship is checked, and then a return flight plan is made.
Disclosure of Invention
The invention aims to provide a method for calculating the operation amount of a water surface cleaning ship, which aims to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme:
a method for calculating the operation amount of a water surface cleaning ship comprises an internal standard ruler arranged in a garbage bin of the water surface cleaning ship and an image acquisition device for acquiring a real-time image of the standard ruler, and comprises the following steps:
step 1, acquiring a real-time image of a scale through an image acquisition device;
step 2, calculating the real-time height of the garbage in the garbage bin according to the real-time image obtained in the step 1;
and 3, calculating the quantity of the stored garbage in the garbage bin according to the real-time height of the garbage in the garbage bin obtained in the step 2 and the size of the garbage bin.
As a further scheme of the invention: the scale is arranged perpendicular to the bottom plate of the garbage bin.
As a further scheme of the invention: the step 2 of obtaining the real-time height of the garbage in the garbage bin through the real-time image comprises the following steps:
step 2.1, inputting the acquired real-time image and selecting an image ROI;
2.2, filtering, denoising and enhancing the selected image;
step 2.3, carrying out edge detection on the processed image and acquiring a digital rectangular frame in the image;
step 2.4, judging the acquired digital rectangular frame, if the acquired digital rectangular frame does not have an effective scale image, executing step 2.1, and if the acquired digital rectangular frame has an effective scale image, executing step 2.5;
step 2.5, identifying the image in the digital rectangular frame and acquiring the scale marks of the identified ruler;
and 2.6, calculating the real-time height of the garbage in the garbage bin according to the obtained scale marks.
As a further scheme of the invention: the filtering formula adopted by the image filtering in the step 2.2 is as follows:
Figure 834200DEST_PATH_IMAGE001
wherein:ffor the original input image，hIn order to output the image after the noise removal,c(t,x) Measure the neighborhood center pointxAnd neighboring pointstThe geometric proximity of (a) to (b),k d are normalized parameters.
As a further scheme of the invention: the formula adopted in the noise reduction and enhancement processing in the step 2.2 is as follows:
Figure 401448DEST_PATH_IMAGE002
wherein gamma is epsilon [0,1]F (x, y) represents an original gray image, g (x, y) represents a linear transformation image, and Mg and Mf are gray coefficientsa、bIs constant, when the image is a 256-term grayscale image, mg = Mf =255,a=500，b=80。
as a further scheme of the invention: the edge detection of the step 2.3 uses a Sobel operator to carry out edge detection, and uses two 3 multiplied by 3 convolution matrixes to detect vertical and horizontal edges;
Figure 558760DEST_PATH_IMAGE003
Figure 477037DEST_PATH_IMAGE004
after the image is subjected to edge detection, the image is corrected by a method of automatically calculating an affine transformation matrix through a Space Transformation Network (STN);
each output pixel may be computed by a sampling kernel centered at a particular location in the input feature map, using an affine point-by-point transform:
Figure 112418DEST_PATH_IMAGE005
Figure 3014DEST_PATH_IMAGE006
and
Figure 596806DEST_PATH_IMAGE007
are all normalized to [ -1,1]The coordinates of the transformed target point and the source point, respectively,
Figure 736800DEST_PATH_IMAGE008
is an affine transformation matrix, i.e. the output of the positioning network, and after conversion, the values at specific pixels in the output V are obtained by applying the source coordinates in the input feature map:
Figure 441451DEST_PATH_IMAGE009
wherein
Figure 186553DEST_PATH_IMAGE010
And
Figure 685668DEST_PATH_IMAGE011
is a general sampling kernel functionk(.) Defines a bilinear image interpolation,
Figure 578537DEST_PATH_IMAGE012
indicates the position (n，m) The value in the input channel c is,
Figure 758983DEST_PATH_IMAGE013
indicating a channelcMiddle position
Figure 420908DEST_PATH_IMAGE014
Processing pixeliThe output value of (1).
As a further scheme of the invention: the detection method of the digital rectangular frame in the step 2.3 comprises the following steps:
first, the sliding window is used to move in four directions, the gray level change in the four directions is calculated, and the local maximum value is positionedE(x，y) The distribution of the homogeneous gradation, in each direction,Eis very small, at edge positions the minimum depends on the edge direction, at corner positions, of each directionEThe change being greater, i.e.EThe 4 directions with the smallest change in value are tested, and if it is still greater than a certain preset threshold, it can be considered as a corner, and the corner calculation formula is as follows:
Figure 356503DEST_PATH_IMAGE015
whereinWAs a result of the sliding window, the window,Iis a value of a gray-scale value,Tis a preset threshold value, ifE(x，y)>TThen point (x，y) Being a corner point.
As a further scheme of the invention: step 2.4, firstly, positioning the candidate rectangular frame, wherein the corners of the scale number frame area are distributed more densely and regularly than other areas, and few corner points appear in the non-rectangular area, calculating the density of the corner points in the sliding window, if the window slides to the rectangular area where the scale numbers are located, the number of the corner points in the window certainly exceeds a preset threshold value, and the positioning of the candidate rectangular frame comprises the following steps:
step 2.41, initializing a sliding window W and an angular point threshold value T;
step 2.42, starting from the left side and the bottom of the image, moving the sliding window and calculating the angular point number in the window bodyc；
Step 2.43, ifc>TThen all corners within the window are represented
Figure 674352DEST_PATH_IMAGE016
Selecting
Figure 720806DEST_PATH_IMAGE017
Sliding the window three times successively upwards and to the right in sequence, and if the number of corners examined in the window exceeds a threshold value, pointing the point
Figure 237238DEST_PATH_IMAGE018
The left lower corner of the candidate rectangular area is taken, and the step 2.45 is carried out;
step 2.44, if the number of corners examined within the window does not exceed the threshold, then the region will be a pseudo-rectangle, returning toGo back to step 2.43, slide window until finding point
Figure 281417DEST_PATH_IMAGE019
；
Step 2.45, confirm point
Figure 883300DEST_PATH_IMAGE018
After, according to the point
Figure 733444DEST_PATH_IMAGE018
Slide window to right to find lower right point
Figure 104382DEST_PATH_IMAGE020
And upper left point
Figure 319463DEST_PATH_IMAGE021
And upper right boundary point
Figure 408642DEST_PATH_IMAGE022
Similarly, thereby obtaining candidate rectangular regions;
step 2.46, scanning the whole image, and repeating the steps to obtain all candidate rectangular areas;
after all candidate areas are obtained, filtering the obtained candidate areas, rejecting the areas if the length and the width of the candidate rectangles obviously exceed the limit range according to the scale size of the known rectangles, and ordering all corner point coordinates from small to large in the candidate rectangular areas needing further verification because the noise and the filtered candidate rectangular areas are possibly smaller than the limit rangeXAndYand is recorded as:
Figure 796898DEST_PATH_IMAGE023
setting a threshold T min And T max In a rectangular areaXDirection andYthe distance boundary of two adjacent angles in the direction is T min And T max Namely:
Figure 225605DEST_PATH_IMAGE024
if the condition is satisfied, the region is regarded as a rectangular region, otherwise, the region is filtered out.
As a further scheme of the invention: 2.5, adopting an OCR digital recognition method for image recognition in the digital rectangular frame, obtaining the scale marks, and calculating the actual height of the garbage in the garbage bin and the actual height of the garbage in the garbage bin according to the scale marksIs calculated by the formula：
Figure 408325DEST_PATH_IMAGE025
Figure 250379DEST_PATH_IMAGE026
h r =d*m
Wherein, the ruler is vertically and evenly distributed withnA scale, each scale isdThe length of each centimeter of the sample is equal to the length of each centimeter,m isRecognized number scale， lThe length of the garbage bin is the length of the garbage bin,h c the distance between the image acquisition device and the bottom of the garbage bin.
As a further scheme of the invention: the volume of the garbage stored in the garbage bin in the step 3 comprises the volume and the mass of the garbage salvaged and stored in the garbage bin, and the volume and the mass of the salvaged and stored garbage are respectively as follows:
volume of garbage:
Figure 176747DEST_PATH_IMAGE027
the quality of the garbage:
Figure 459960DEST_PATH_IMAGE028
wherein w is the width of the garbage bin,Sthe area of the bottom surface of the garbage bin is S = l × w.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, the scale is arranged in the garbage bin, the image of the scale is acquired in real time or at a certain time interval through the image acquisition device, the scale image is identified, the scale is arranged on the scale, the stacked garbage is a part of a buried mark, the depth of the garbage is judged according to the identified scale on the scale, and then the storage amount of the garbage in the garbage bin can be calculated.
Drawings
FIG. 1 is a schematic flow chart of a method for calculating the workload of a clean-keeping ship according to this embodiment;
fig. 2 is a schematic diagram of the present embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, in an embodiment of the present invention, a method for calculating an operation amount of a water surface cleaning ship includes an internal scale disposed in a garbage bin of the water surface cleaning ship and an image acquisition device for acquiring a real-time image of the scale, where the scale is disposed perpendicular to a bottom plate of the garbage bin, and the method for calculating an operation amount of a water surface cleaning ship includes the following steps:
step 1, acquiring a real-time image of a scale through an image acquisition device, wherein in the embodiment, the image acquisition device is a video camera or a camera, and the image of the scale is acquired through the video camera or the camera;
step 2, calculating the real-time height of the garbage in the garbage bin according to the real-time image obtained in the step 1;
the step 2 of acquiring the real-time height of the garbage in the garbage bin through the real-time image comprises the following steps:
step 2.1, inputting the acquired real-time image and selecting an image ROI;
2.2, filtering, denoising and enhancing the selected image;
the filtering formula adopted by the image filtering is as follows:
Figure 344740DEST_PATH_IMAGE001
wherein:fis an original input image and is used as a reference image,hin order to output the image after the noise removal,c(t,x) Measure the neighborhood center pointxAnd neighboring pointstThe geometric proximity of (a) to (b),k d is a normalized parameter;
the scale image is denoised and contrast enhanced in an image range, because the scale image background is usually complex, and because factors such as weather conditions, illumination, scale fouling and the like exist, uncertain factors exist in the scale image, aiming at the problems, a grayscale-based denoising and image enhancement algorithm is adopted, a scale region is enhanced and other regions in the image are eliminated through the following formula, and the formula adopted for denoising and enhancing is as follows:
Figure 142932DEST_PATH_IMAGE029
wherein gamma epsilon [0,1]F (x, y) represents an original gray image, g (x, y) represents a linear transformation image, and Mg and Mf are gray coefficientsa、bIs constant, when the image is a 256-term grayscale image, mg = Mf =255,a=500，b=80；
step 2.3, carrying out edge detection on the processed image and acquiring a digital rectangular frame in the image;
2.3, performing edge detection by using a Sobel operator, and detecting vertical and horizontal edges by using two 3 multiplied by 3 convolution matrixes;
Figure 810673DEST_PATH_IMAGE030
Figure 16570DEST_PATH_IMAGE004
since the number of pictures is likely to be skewed, the pictures need to be corrected for improved recognition accuracy, the input images are obtained by a method in which an affine transformation matrix is automatically calculated by a Spatial Transformation Network (STN), invariance to translation, scaling, rotation and more general deformations is learned by the STN, the spatial transformer module is a dynamic mechanism that can actively spatially transform an image (or feature map) by generating an appropriate transformation for each input sample, and the spatial transformer module combines a positioning network and a sampling mechanism. Input to the positioning network is a profileUAnd regression theta, theta = f is performed on the transformation parameters loc (U), each output pixel may be computed by a sampling kernel centered at a particular location in the input feature map, employing an affine point-by-point transformation;
after the image is subjected to edge detection, correcting the image by a method of automatically calculating an affine transformation matrix through a Space Transformation Network (STN);
each output pixel may be computed by a sampling kernel centered at a particular location in the input feature map, using an affine point-by-point transform:
Figure 72250DEST_PATH_IMAGE005
Figure 92159DEST_PATH_IMAGE006
and
Figure 829171DEST_PATH_IMAGE007
are all normalized to [ -1,1]Are respectively a changeThe coordinates of the target point and the source point are changed,
Figure 149294DEST_PATH_IMAGE031
is an affine transformation matrix, i.e. the output of the positioning network, and after conversion, the values at specific pixels in the output V are obtained by applying the source coordinates in the input feature map:
Figure 47980DEST_PATH_IMAGE032
wherein
Figure 820764DEST_PATH_IMAGE010
And
Figure 158204DEST_PATH_IMAGE011
is a general sampling kernel functionk(.) Defines a bilinear image interpolation,
Figure 67254DEST_PATH_IMAGE012
indicating a location (n，m) The value in the input channel c is,
Figure 402421DEST_PATH_IMAGE013
indicating a channelcMiddle position
Figure 662501DEST_PATH_IMAGE014
At pixeliThe output value of (d);
the space transformer is combined with the convolutional neural network, the representation capability of the network is effectively improved, the identification precision of the convolutional neural network is improved, the inclined image and the normal image are used for training, the mapping relation between the two images is automatically searched, and the space transformer is combined with a digital identification algorithm. The space transformer can affine the whole scale image and several numbers on the affine scale
The detection method of the digital rectangular frame comprises the following steps:
first using a sliding windowThe mouth is moved in four directions, the gray changes in the four directions are calculated, and local maxima are locatedE(x，y) The distribution of the homogeneous gradation, in each direction,Eis very small, at edge positions the minimum depends on the edge direction, at corner positions, of each directionEThe change being greater, i.e.EThe 4 directions with the smallest change in value are tested, and if it is still greater than a certain preset threshold, it can be considered as a corner, and the corner calculation formula is as follows:
Figure 741315DEST_PATH_IMAGE033
whereinWAs a result of the sliding window, the window,Iis a gray-scale value that is,Tis a preset threshold value, ifE(x，y)>TThen point (x，y) Is a corner point
Step 2.4, judging the acquired digital rectangular frame, if the acquired digital rectangular frame does not have an effective scale image, executing step 2.1, and if the acquired digital rectangular frame has an effective scale image, executing step 2.5;
firstly, positioning a candidate rectangular frame, wherein the corners of a scale number frame area are distributed more densely and regularly than other areas, and few corner points appear in a non-rectangular area, calculating the density of the corner points in a sliding window, if the window slides to the rectangular area where scale numbers are located, the number of the corner points in the window certainly exceeds a preset threshold value, and the positioning of the candidate rectangular frame comprises the following steps:
step 2.41, initializing a sliding window W and an angular point threshold value T;
step 2.42, starting from the left side and the bottom of the image, moving the sliding window and calculating the angular point number in the window bodyc；
Step 2.43, ifc>TThen all corners within the window are represented
Figure 36030DEST_PATH_IMAGE016
Selecting
Figure 73256DEST_PATH_IMAGE017
Sliding the window three times successively upwards and to the right in sequence, and if the number of corners examined in the window exceeds a threshold value, pointing the point
Figure 23895DEST_PATH_IMAGE018
The left lower corner of the candidate rectangular area is taken, and the step 2.45 is carried out;
step 2.44, if the number of corners examined in the window does not exceed the threshold, the region will be a pseudo-rectangle, return to step 2.43, slide the window until a point is found
Figure 703138DEST_PATH_IMAGE019
；
Step 2.45, confirm point
Figure 586780DEST_PATH_IMAGE018
After, according to the point
Figure 998170DEST_PATH_IMAGE018
Slide window to right to find lower right point
Figure 967263DEST_PATH_IMAGE020
And upper left point
Figure 450197DEST_PATH_IMAGE021
And upper right boundary point
Figure 391608DEST_PATH_IMAGE022
Similarly, thereby obtaining candidate rectangular regions;
step 2.46, scanning the whole image, and repeating the steps to obtain all candidate rectangular areas;
after all candidate areas are obtained, filtering the obtained candidate areas, rejecting the areas if the length and the width of the candidate rectangles obviously exceed the limit range according to the scale size of the known rectangles, and ordering all corner point coordinates from small to large in the candidate rectangular areas needing further verification because the noise and the filtered candidate rectangular areas are possibly smaller than the limit rangeXAndYand is recorded as:
Figure 301795DEST_PATH_IMAGE023
setting a threshold T min And T max In a rectangular areaXDirection andYthe distance boundary of two adjacent angles in the direction is T min And T max Namely:
Figure 758184DEST_PATH_IMAGE024
if the condition is met, the region is regarded as a rectangular region, and if not, the region is filtered;
an OCR (optical character recognition) digital recognition method is adopted for image recognition in the digital rectangular frame, wherein OCR is optical character recognition, and a large number of scale digital pictures are acquired on site and stored in a template through OCR pre-training. The OCR stage algorithm is stored in a database in advance by calculating the mean value of the image, is completed by solving the correlation coefficient during detection, and has the characteristics of high running speed, high precision and the like.
When a stream of image pixels is obtained, the average of the image will be calculated and the image will be stored in a variable on a pixel-by-pixel basis. Template matching is achieved by solving for correlation coefficients, and template matching techniques match an input image with a template image to identify digital characters. The template images are stored in a database and trained from a dataset collected on-site. The correlation coefficient ρ is calculated as follows:
Figure 716913DEST_PATH_IMAGE034
wherein, the first and the second end of the pipe are connected with each other,A、Bfor the matrix of images to be matched,
Figure 575148DEST_PATH_IMAGE035
and
Figure 390657DEST_PATH_IMAGE036
are mean values.
After the scale marks are obtained, the actual height of the garbage in the garbage bin is calculated according to the scale marks, and the calculation formula of the actual height of the garbage in the garbage bin is as follows:
Figure 334342DEST_PATH_IMAGE037
Figure 96762DEST_PATH_IMAGE038
h r =d*m
wherein, the ruler is vertically and evenly distributed withnA scale, each scale isdThe length of each centimeter of the product is,m isRecognized number scale， lThe length of the garbage bin is the length of the garbage bin,h c the distance between the image acquisition device and the bottom of the garbage bin
Step 2.6, calculating the real-time height of the garbage in the garbage bin according to the obtained scale marks
Step 3, calculating the quantity of the stored garbage in the garbage bin according to the real-time height of the garbage in the garbage bin obtained in the step 2 and the size of the garbage bin;
the volume of rubbish under-deck storage includes volume and the quality of the rubbish of salvage storage in the rubbish under-deck, and the volume and the quality of the rubbish of salvage and storage are respectively:
volume of garbage:
Figure 809503DEST_PATH_IMAGE039
the quality of the garbage is as follows:
Figure 61493DEST_PATH_IMAGE040
wherein w is the width of the garbage bin,Sthe area of the bottom surface of the garbage bin is S = l × w.
In the embodiment, the garbage bin has the length and width dimensionsl、wRespectively 200 cm and 100 cm, according to the garbageThe size of the cabin determines the installation positions of the scale plate and the camera, and the installation height of the camerah c =50 cm, the scale plate of the scale is arranged opposite to the camera, and the scale plate is vertically and uniformly distributed withn=100 scales, each scale beingd=2 cm, if the digital scales 100, 99, … …, 51 are recognized, the current camera sight line trash height position is consideredh r =d*mAnd if the distance is not less than 2 x 50 cm and the distance is not less than 100 cm, the included angle between the visual line direction of the camera and the horizontal direction is formed
Figure 492474DEST_PATH_IMAGE041
Theta is about 14 deg. and thus the actual height of the refuse
Figure 793005DEST_PATH_IMAGE042
Centimeter, and further can estimate that the volume of salvaged garbage is about
Figure 360253DEST_PATH_IMAGE043
And cubic meter, and finally displaying the garbage salvage quantity output by the algorithm on a mobile phone APP, a webpage or a client in real time.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present specification describes embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and it is to be understood that all embodiments may be combined as appropriate by one of ordinary skill in the art to form other embodiments as will be apparent to those of skill in the art from the description herein.

Claims (7)

1. A method for calculating the operation quantity of a water surface cleaning ship is characterized by comprising an internal standard ruler arranged in a garbage bin of the water surface cleaning ship and an image acquisition device for acquiring a real-time image of the standard ruler, wherein the method for calculating the operation quantity of the water surface cleaning ship comprises the following steps:
step 1, acquiring a real-time image of a scale through an image acquisition device;
step 2, calculating the real-time height of the garbage in the garbage bin according to the real-time image obtained in the step 1;
the step 2 of obtaining the real-time height of the garbage in the garbage bin through the real-time image comprises the following steps:
step 2.1, inputting the acquired real-time image and selecting an image ROI;
2.2, filtering, denoising and enhancing the selected image;
step 2.3, performing edge detection on the processed image and acquiring a digital rectangular frame in the image;
the detection method of the digital rectangular frame in the step 2.3 comprises the following steps:
firstly, moving in four directions by using a sliding window, calculating the gray scale change in the four directions, and positioning a local maximum valueE(x，y) The distribution of the homogeneous gradation, in each direction,Eis very small, at edge positions the minimum depends on the edge direction, at corner positions, of each directionEThe change is larger, i.e.EThe 4 directions with the smallest change in value are tested, and if it is still greater than a certain preset threshold, it can be considered as a corner, and the corner calculation formula is as follows:
Figure 534105DEST_PATH_IMAGE001
whereinWAs a result of the sliding window, the window,Iis a gray-scale value that is,Tis a preset threshold value, ifE(x，y)>T, then point (x)，y) is an angular point;
step 2.4, judging the acquired digital rectangular frame, if the acquired digital rectangular frame does not have an effective scale image, executing step 2.1, and if the acquired digital rectangular frame has an effective scale image, executing step 2.5;
step 2.4, firstly, positioning the candidate rectangular frame, wherein the corners of the scale number frame area are distributed more densely and regularly than other areas, and few corner points appear in the non-rectangular area, calculating the density of the corner points in the sliding window, if the window slides to the rectangular area where the scale numbers are located, the number of the corner points in the window certainly exceeds a preset threshold value, and the positioning of the candidate rectangular frame comprises the following steps:
step 2.41, initializing a sliding window W and an angular point threshold value T;
step 2.42, starting from the left side and the bottom of the image, moving the sliding window and calculating the angular point number in the window bodyc；
Step 2.43, ifc>TThen all corners within the window are represented
Figure 72533DEST_PATH_IMAGE002
Selecting
Figure 535876DEST_PATH_IMAGE003
Sliding the window three times successively upwards and to the right in sequence, and if the number of corners examined in the window exceeds a threshold value, pointing the point
Figure 270482DEST_PATH_IMAGE004
The left lower corner of the candidate rectangular area is taken, and the step 2.45 is carried out;
step 2.44, if the number of corners checked in the window does not exceed the threshold, the area will be a pseudo-rectangle, return to step 2.43, slide the window until a point is found
Figure 299618DEST_PATH_IMAGE005
；
Step 2.45, confirm point
Figure 477790DEST_PATH_IMAGE006
After, according to the point
Figure 959587DEST_PATH_IMAGE006
Sliding window to right to find lower right point
Figure 264928DEST_PATH_IMAGE007
And upper left point
Figure 148571DEST_PATH_IMAGE008
And upper right boundary point
Figure 497644DEST_PATH_IMAGE009
Similarly, a candidate rectangular region is obtained;
step 2.46, scanning the whole image, and repeating the steps to obtain all candidate rectangular areas;
after all candidate areas are obtained, filtering the obtained candidate areas, rejecting the areas if the length and the width of the candidate rectangles obviously exceed the limit range according to the scale size of the known rectangles, and ordering all corner point coordinates from small to large in the candidate rectangular areas needing further verification because the noise and the filtered candidate rectangular areas are possibly smaller than the limit rangeXAndYand is recorded as:
Figure 669999DEST_PATH_IMAGE010
setting a threshold T min And T max In a rectangular areaXDirection andYthe distance boundary of two adjacent angles in the direction is T min And T max Namely:
Figure 621774DEST_PATH_IMAGE011
if the condition is met, the region is regarded as a rectangular region, and if not, the region is filtered;
step 2.5, identifying the image in the digital rectangular frame and acquiring the scale marks of the identified ruler;
step 2.6, calculating the real-time height of the garbage in the garbage bin according to the obtained scale marks;
and 3, calculating the quantity of the stored garbage in the garbage bin according to the real-time height of the garbage in the garbage bin obtained in the step 2 and the size of the garbage bin.
2. The method for calculating the working capacity of the water surface cleaning ship according to claim 1, wherein the scale is arranged perpendicular to the bottom plate of the garbage bin.
3. The method for calculating the operation quantity of the water surface cleaning ship according to claim 1, wherein the image filtering in the step 2.2 adopts a filtering formula as follows:
Figure 15716DEST_PATH_IMAGE012
wherein:fis an original input image and is used as a reference image,hin order to output the image after the noise removal,c(t,x) Measure the neighborhood center pointxAnd neighboring pointstThe geometric proximity of (a) to (b),k d are normalized parameters.
4. The method for calculating the operation amount of the water surface cleaning ship according to claim 1, wherein the formula adopted in the noise reduction and enhancement treatment in the step 2.2 is as follows:
Figure 660324DEST_PATH_IMAGE013
wherein gamma is epsilon [0,1]F (x, y) represents an original gray image, g (x, y) represents a linear transformation image, and Mg and Mf are gray coefficientsa、bIs constant, when the image is a 256-term grayscale image, mg = Mf =255,a=500，b=80。
5. the method for calculating the work capacity of the water surface cleaning ship according to claim 1, wherein the edge detection of the step 2.3 uses a Sobel operator to perform edge detection, and uses two 3 x 3 convolution matrixes to detect vertical and horizontal edges;
Figure 992079DEST_PATH_IMAGE014
Figure 950808DEST_PATH_IMAGE015
after the edge detection is carried out on the image, the image is corrected by a method of automatically calculating an affine transformation matrix through a Space Transformation Network (STN);
each output pixel may be computed by a sampling kernel centered at a particular location in the input feature map, using an affine point-by-point transform:
Figure 543463DEST_PATH_IMAGE016
Figure 247720DEST_PATH_IMAGE017
and
Figure 191406DEST_PATH_IMAGE018
are all normalized to [ -1,1]The coordinates of the transformed target point and the source point, respectively,
Figure 625929DEST_PATH_IMAGE019
is an affine transformation matrix, i.e. the output of the positioning network, and after conversion, the values at specific pixels in the output V are obtained by applying the source coordinates in the input feature map:
Figure 541933DEST_PATH_IMAGE020
wherein
Figure 262764DEST_PATH_IMAGE021
And
Figure 83958DEST_PATH_IMAGE022
is a general sampling kernel functionk(.) Defines a bilinear image interpolation,
Figure 181227DEST_PATH_IMAGE023
indicates the position (n，m) The value in the input channel c is,
Figure 217316DEST_PATH_IMAGE024
indicating a channelcMiddle position
Figure 984415DEST_PATH_IMAGE025
At pixeliThe output value of (1).
6. The method for calculating the operation quantity of the water surface cleaning ship according to claim 1, wherein the step 2.5 adopts an OCR digital recognition method for image recognition in the digital rectangular frame, after the scale marks are obtained, the actual height of the garbage in the garbage bin is calculated according to the scale marks, and the calculation formula of the actual height of the garbage in the garbage bin is as follows:
Figure 168272DEST_PATH_IMAGE026
Figure 695330DEST_PATH_IMAGE027
h r =d*m
wherein, the rulers are all verticalIs uniformly distributed withnEach scale is a scaledThe length of each centimeter of the product is,mfor identifying a numerical scale， lThe length of the garbage bin is the length of the garbage bin,h c the distance between the image acquisition device and the bottom of the garbage bin.
7. The method for calculating the operation amount of the water surface cleaning ship according to claim 6, wherein the amount of the garbage stored in the garbage bin in the step 3 comprises the volume and the mass of the garbage salvaged and stored in the garbage bin, and the volume and the mass of the salvaged and stored garbage are respectively as follows:
volume of garbage:
Figure 117084DEST_PATH_IMAGE028
the quality of the garbage is as follows:
Figure 320664DEST_PATH_IMAGE029
wherein w is the width of the garbage bin,Sthe area of the bottom surface of the garbage bin is S = l × w.
CN202210921088.5A 2022-08-02 2022-08-02 Method for calculating operation amount of water surface cleaning ship Active CN114973261B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210921088.5A CN114973261B (en) 2022-08-02 2022-08-02 Method for calculating operation amount of water surface cleaning ship

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210921088.5A CN114973261B (en) 2022-08-02 2022-08-02 Method for calculating operation amount of water surface cleaning ship

Publications (2)

Publication Number Publication Date
CN114973261A CN114973261A (en) 2022-08-30
CN114973261B true CN114973261B (en) 2022-11-11

Family

ID=82968569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210921088.5A Active CN114973261B (en) 2022-08-02 2022-08-02 Method for calculating operation amount of water surface cleaning ship

Country Status (1)

Country Link
CN (1) CN114973261B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102975826A (en) * 2012-12-03 2013-03-20 上海海事大学 Portable ship water gauge automatic detection and identification method based on machine vision
CN104677456A (en) * 2015-02-10 2015-06-03 厦门精图信息技术股份有限公司 Water level monitoring method based on computer image recognition, and water gauge and monitoring system of water level monitoring method
CN108426618A (en) * 2018-06-20 2018-08-21 深圳亿腾物联智能技术有限责任公司 A kind of silo stock amount detection device and method
CN109993166A (en) * 2019-04-03 2019-07-09 同济大学 The readings of pointer type meters automatic identifying method searched based on scale
CN110717924A (en) * 2019-09-27 2020-01-21 中国检验认证集团河北有限公司黄骅港分公司 Ship water gauge marking recognition method and device, computer equipment and storage medium
CN111062985A (en) * 2019-12-27 2020-04-24 苏州环境云信息科技有限公司 Vehicle-mounted swill recovery volume measurement system and method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102975826A (en) * 2012-12-03 2013-03-20 上海海事大学 Portable ship water gauge automatic detection and identification method based on machine vision
CN104677456A (en) * 2015-02-10 2015-06-03 厦门精图信息技术股份有限公司 Water level monitoring method based on computer image recognition, and water gauge and monitoring system of water level monitoring method
CN108426618A (en) * 2018-06-20 2018-08-21 深圳亿腾物联智能技术有限责任公司 A kind of silo stock amount detection device and method
CN109993166A (en) * 2019-04-03 2019-07-09 同济大学 The readings of pointer type meters automatic identifying method searched based on scale
CN110717924A (en) * 2019-09-27 2020-01-21 中国检验认证集团河北有限公司黄骅港分公司 Ship water gauge marking recognition method and device, computer equipment and storage medium
CN111062985A (en) * 2019-12-27 2020-04-24 苏州环境云信息科技有限公司 Vehicle-mounted swill recovery volume measurement system and method

Also Published As

Publication number Publication date
CN114973261A (en) 2022-08-30

Similar Documents

Publication Publication Date Title
CN109443480B (en) Water level scale positioning and water level measuring method based on image processing
CN112419250A (en) Pavement crack digital image extraction, crack repair and crack parameter calculation method
CN105588845B (en) A kind of weld defect characteristic parameter extraction method
CN113240626B (en) Glass cover plate concave-convex type flaw detection and classification method based on neural network
CN112233116B (en) Concave-convex mark visual detection method based on neighborhood decision and gray level co-occurrence matrix description
CN111915704A (en) Apple hierarchical identification method based on deep learning
CN110473221B (en) Automatic target object scanning system and method
CN111257348B (en) LED light guide plate defect detection method based on machine vision
CN111027446B (en) Coastline automatic extraction method of high-resolution image
CN108470338B (en) A kind of water level monitoring method
CN111652213A (en) Ship water gauge reading identification method based on deep learning
CN112767359B (en) Method and system for detecting corner points of steel plate under complex background
CN114549981A (en) Intelligent inspection pointer type instrument recognition and reading method based on deep learning
CN114240845B (en) Light cutting method surface roughness measurement method applied to cutting workpiece
CN113034399A (en) Binocular vision based autonomous underwater robot recovery and guide pseudo light source removing method
CN111652844B (en) X-ray defect detection method and system based on digital image region growing
CN113674206B (en) Extraction method suitable for characteristic parameters of deep-melting K-TIG welding molten pool and keyhole entrance
CN113034474A (en) Test method for wafer map of OLED display
CN116152115B (en) Garbage image denoising processing method based on computer vision
CN109410139A (en) Inside a kind of historical relic and surface disease digital assay appraisal procedure
CN109829858A (en) A kind of shipborne radar image spilled oil monitoring method based on local auto-adaptive threshold value
CN106780718A (en) A kind of three-dimensional rebuilding method of paleontological fossil
CN111354047B (en) Computer vision-based camera module positioning method and system
CN114973261B (en) Method for calculating operation amount of water surface cleaning ship
CN114529715B (en) Image identification method and system based on edge extraction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant