CN113305858B - Visual robot method and device for removing shellfish in raw water pipeline - Google Patents
Visual robot method and device for removing shellfish in raw water pipeline Download PDFInfo
- Publication number
- CN113305858B CN113305858B CN202110631864.3A CN202110631864A CN113305858B CN 113305858 B CN113305858 B CN 113305858B CN 202110631864 A CN202110631864 A CN 202110631864A CN 113305858 B CN113305858 B CN 113305858B
- Authority
- CN
- China
- Prior art keywords
- shellfish
- robot
- pipeline
- binocular
- visual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/008—Manipulators for service tasks
- B25J11/0085—Cleaning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Cleaning In General (AREA)
Abstract
The invention discloses a visual robot method and a device for removing shellfish in a raw water pipeline, which comprises a robot, wherein a cleaning device is arranged on a driving arm of the robot, a crushing device and a binocular visual device are respectively arranged at the front end of the cleaning device, the device also comprises a controller, the binocular visual device is in communication connection with the controller, and the controller is in communication connection with the robot, the cleaning device and the crushing device respectively.
Description
Technical Field
The invention particularly relates to a visual robot method and a device for removing shellfish in a raw water pipeline.
Background
The number station of the Chinese dams accounts for 50% of the total amount of the world, deposits of shellfish activity and shellfish debris exist on the inner surfaces of part of water conveying pipelines and water taking pipelines, if the deposits are excessively superposed, water conveying in the pipelines is affected, and the water source is polluted by dead shellfish. The shellfish with strong adsorption force is adsorbed on the inner wall of the pipeline, so that people can hardly find the shellfish from the outside, and countless shellfish with superposed activity have large adsorption force and other impurities are difficult to remove in pipeline decontamination. In order to reduce the shellfish stacked and grown on the inner wall of the pipeline, people take different measures, including eating shellfish, putting a certain proportion of medicine to kill the shellfish, and manually cleaning. These methods are not effective and still result in contamination by drug kill and uncleaned deposits such as dead shellfish. Although there are some pipeline cleaning robots at home and abroad, the stacking volume of shellfish objects is not accurately calculated, and the vertical distance between the stacking objects and the pipeline is changed to the maximum and along the pipe wall so as to determine the optimal tool center and the target center through detection calculation to effectively clean the stacking bodies. The invention is still significant for the visual accurate identification and positioning method, the robot and the device innovation for removing the active shellfish and the remains thereof.
Therefore, a visual robot method and a device for removing shellfish in the raw water pipeline are urgently needed to solve the problem.
Disclosure of Invention
The invention aims to provide a visual detection system for biomimetically identifying sundries such as shellfish and debris, an image processing algorithm and a method for calculating the blocking size of the shellfish and debris.
In order to achieve the aim of the invention, the visual robot method for removing the shellfish in the raw water pipeline comprises the following steps:
1) performing binocular calibration and photo adjustment by using a binocular vision device;
2) acquiring shellfish and sundry image data by using a binocular vision device, and transmitting the data to a controller;
3) training a neural network system for identifying, positioning and navigating by using the data through a controller to perform image semantic segmentation network training of a Mask RCNN model, performing edge calculation and point cloud classification network training, so that the trained neural network system for visual navigation can identify the spatial position of the shellfish in the arc-shaped pipeline;
4) acquiring an image in the pipeline: controlling the movement and positioning of the robot and the cleaning device through a controller;
5) searching a part with the largest shellfish adsorption accumulation because the pipeline is in a circular stereo, starting a searchlight in a binocular vision device, shooting by using a binocular stereo vision device to obtain shellfish images of a left camera and a right camera at the lower part of the pipeline, and performing binocular correction on the left image and the right image to obtain binocular corrected left and right shellfish images;
6) and image segmentation: segmenting the inside of the pipeline in the left and right shellfish images after binocular correction by adopting a trained and programmed Mask RCNN model;
7) measuring the volume of shellfish sundries accumulated on the 360-degree surface of the pipeline at the closest distance by a vision camera;
performing three-dimensional reconstruction on the left and right shellfish images by using a stereo matching algorithm to obtain three-dimensional point cloud;
then accurately finding the three-dimensional point cloud of the shellfish stacked in the pipeline by using the model, finding the maximum and minimum volumes of the stacked shellfish, and calculating the central coordinate of the shellfish stack with the maximum volume;
8) based on the visual positioning of shellfish deposits, the robot uses a crushing device to drill and ream the superposed shellfish sundries absorbed in the pipeline;
9) and taking out the crushed slag from the raw water pipe by using a cleaning device to finish cleaning.
Preferably, in step 1, the binocular vision device firstly calibrates a single eye to obtain an internal parameter matrix and a distortion coefficient matrix of each camera; performing binocular calibration on the binocular vision device to obtain a reprojection matrix for three-dimensional reconstruction and a unit conversion relation between the pixel distance and the millimeter distance; aiming at the brightness inside the road, the LED illumination is adjusted, the optimal illumination is optimized, and the noise caused by illumination is reduced.
Preferably, in the step 2, a plurality of images of shellfish sundries inside and outside different pipelines are acquired by a vision camera, programming is carried out, and automatic marking is realized.
Preferably, when the mechanical arm of the robot is pushed forwards, the barbs can pull the crushed shellfish and crushed slag with small volume backwards to the shovel, and then the crushed shellfish and crushed slag are conveyed to the outside through the conveying screw.
In order to better realize the visual robot method for removing the shellfish in the raw water pipeline, the invention also provides a visual robot device for removing the shellfish in the raw water pipeline, which comprises a robot, wherein a cleaning device is arranged on a driving arm of the robot, the front end of the cleaning device is respectively provided with a crushing device and a binocular visual device, the visual robot device further comprises a controller, the binocular visual device is in communication connection with the controller, and the controller is respectively in communication connection with the robot, the cleaning device and the crushing device.
Preferably, the binocular vision device comprises two vision cameras and a searchlight.
Preferably, the cleaning device comprises a motor and a transmission screw arranged on the motor.
Preferably, the front part of the conveying screw is also provided with a barb for clearing the slag.
The invention has the advantages of convenient and reasonable control, high cleaning efficiency and strong practicability, and compared with the prior art, the invention can automatically clean shellfish and remains thereof in the raw water pipe, realize automatic identification, crushing and cleaning, improve efficiency and reduce pollution.
Drawings
FIG. 1 is a flow chart of shellfish sundry algorithm and cut-off path planning in the present invention;
FIG. 2 is a flow chart of shellfish sundry algorithm and cleaning path planning in the present invention;
FIG. 3 is a schematic structural diagram of a vision robot according to the present invention;
fig. 4 is a sectional view of the raw water pipe.
Detailed Description
The invention is further described with reference to the following figures and examples.
Referring to fig. 1-4, a visual robot device for removing shellfish from a raw water pipeline comprises a robot 1, a cleaning device 2 is arranged on a driving arm of the robot 1, a crushing device 3 and a binocular visual device 4 are respectively arranged at the front end of the cleaning device 2, and a controller is further included, the binocular visual device 4 is in communication connection with the controller, and the controller is in communication connection with the robot 1, the cleaning device 2 and the crushing device 3 respectively.
The binocular vision device 4 comprises two vision cameras and a searchlight, and is used for image acquisition, shellfish sundry identification, pipeline arc wall identification, obstacle identification, positioning of a copying shellfish removal combined cutter, and robot and distance measurement in the shellfish removal process.
The cleaning device 2 comprises a motor and a transmission screw rod arranged on the motor.
The front part of the transmission screw is also provided with a barb for clearing the slag.
Referring to fig. 4, the small yellow circle with the diameter of small d is the diameter of the cutter in the crushing device 3, and the black is the shellfish sundries with high aggregation and viscosity.
The robot body 1 is composed of a multi-degree-of-freedom mechanical arm and a driving and controlling integrated servo motor, and the mechanical arm can drive the combined tool to move and move in the X, Y and Z directions.
The crushing device 3 comprises a drill bit at the front end and a milling cutter, the diameter of the drill bit at the front end is smaller than the outer diameter of the milling cutter, the milling cutter with three edges at the rear end is in a large-diameter form, spiral grooves are formed in 2 kinds of cutters, and a cutter handle end shaft is designed to be a conveying channel with an edge internally-rolled bionic tooth involute;
the drilling bit or the three-edge milling cutter axis coordinate and the vertical distance between the inner wall of the pipeline are calculated each time through visual detection and calculation of the shellfish accumulation volume, and are sent to a control system, and the drilling bit starts to work through motion planning;
the drilling head drills an opening or a notch on the shellfish stacking upper body which is strongly attached to and stacked into a whole, and then the rotary three-edge milling cutter expands at the opening to cut off more shellfish.
The crushing device 3 moves along the pipeline semicircular arc line, the tool continuously performs feed motion to cut off the shellfish layer of the semicircular arc surface, the cut and superposed shellfish blocks move towards the direction of the pipeline opening along the edge inward-rolling bionic tooth type involute conveying channel by extrusion, and the edge inward-rolling bionic teeth can grab the cut shellfish sundries to prevent the sundries from falling off; when shellfish objects and deposits are cut off along the pipeline for a certain length, the tail end actuating mechanism returns most of shellfish deposits at the opening of the pipeline by moving back, and the shellfish deposits are poured into the sundries box. Then the sliding support and the axis of the tool move forwards along the sleeve to carry out the second cycle operation.
Referring to fig. 1-3, a visual robot method for removing shellfish in a raw water pipeline comprises a visual robot device and further comprises the following steps:
the camera is calibrated (an automatic calibration visual system can be used), after calibration, the initial position of the mechanical arm is set, the position of the visual camera relative to the three-dimensional space is also determined, and simultaneously, the pixel unit of the camera can be converted into the millimeter required by people.
And detecting shellfish sundries in the pipeline by using a visual algorithm under the condition that the diameter of the pipeline is known.
Step 2, acquiring shellfish and sundry image data by using a binocular vision device 4, and transmitting the data to a controller;
step 4, acquiring an image in the pipeline: the movement and the positioning of the robot 1 and the cleaning device 2 are controlled by a controller;
step 5, searching a part with the largest shellfish adsorption accumulation because the pipeline is in a circular stereo mode, starting a searchlight in the binocular vision device 4, shooting by using the binocular stereo vision device to obtain shellfish images of a left camera and a right camera at the lower part of the pipeline, and performing binocular correction on the left image and the right image to obtain binocular-corrected left and right shellfish images;
step 6, image segmentation: segmenting the inside of the pipeline in the left and right shellfish images after binocular correction by adopting a trained and programmed Mask RCNN model;
step 7, measuring the volume of shellfish sundries accumulated on the 360-degree surface of the pipeline at the closest distance by a vision camera;
performing three-dimensional reconstruction on the left and right shellfish images by using a stereo matching algorithm to obtain three-dimensional point cloud;
then accurately finding the three-dimensional point cloud of the shellfish stacked in the pipeline by using the model, finding the maximum and minimum volumes of the stacked shellfish, and calculating the central coordinate of the shellfish stack with the maximum volume;
step 8, positioning the shellfish deposit based on vision, and drilling and twisting the superposed shellfish sundries adsorbed in the pipeline by using the crushing device 3 by the robot;
and 9, taking out the crushed slag from the raw water pipe by using the cleaning device 2 to finish cleaning.
Further, in step 1: the binocular vision device 4 firstly calibrates a single eye to obtain an internal parameter matrix and a distortion coefficient matrix of each camera; performing binocular calibration on the binocular vision device 4 to obtain a reprojection matrix for three-dimensional reconstruction and a unit conversion relation between the pixel distance and the millimeter distance; aiming at the brightness inside the road, the LED illumination is adjusted, the optimal illumination is optimized, and the noise caused by illumination is reduced.
Further, in the step 2, a plurality of images of shellfish sundries inside and outside different pipelines are acquired by using a visual camera, programming is carried out, and automatic marking is realized.
The number of the photos collected by the vision camera is 1500-2000.
Furthermore, when the mechanical arm of the robot is pushed forward, the inverted hook can pull the crushed shellfish and crushed slag with small volume backwards to shovel, and then the crushed shellfish and crushed slag are conveyed to the outside through the conveying screw rod.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are used only for the convenience of description and simplicity of description, rather than to indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the invention, the terms "first" and "second" are used for descriptive purposes only, and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (8)
1. A visual robot method for removing shellfish in a raw water pipeline is characterized in that: the method comprises the following steps:
1) carrying out binocular calibration and adjustment by using a binocular vision device (4);
2) the binocular vision device (4) is used for collecting shellfish and sundry image data and transmitting the data to the controller;
3) training a neural network system for identifying, positioning and navigating by using the data through a controller to perform image semantic segmentation network training of a Mask RCNN model, performing edge calculation and point cloud classification network training, so that the trained neural network system for visual navigation can identify the spatial position of the shellfish in the arc-shaped pipeline;
4) acquiring an image in the pipeline: the movement and the positioning of the robot (1) and the cleaning device (2) are controlled by a controller;
5) searching a part with the largest shellfish adsorption accumulation because the pipeline is in a circular stereo, starting a searchlight in a binocular vision device (4), shooting by using the binocular stereo vision device to obtain shellfish images of a left camera and a right camera at the lower part of the pipeline, and performing binocular correction on the left image and the right image to obtain binocular-corrected left and right shellfish images;
6) and image segmentation: segmenting the inside of the pipeline in the left and right shellfish images after binocular correction by adopting a trained and programmed Mask RCNN model;
7) measuring the volume of shellfish sundries accumulated on the 360-degree surface of the pipeline at the closest distance by a vision camera;
performing three-dimensional reconstruction on the left and right shellfish images by using a stereo matching algorithm to obtain three-dimensional point cloud;
then accurately finding the three-dimensional point cloud of the shellfish stacked in the pipeline by using the model, finding the maximum and minimum volumes of the stacked shellfish, and calculating the central coordinate of the shellfish stack with the maximum volume;
8) based on the visual positioning of shellfish deposits, the robot uses a crushing device (3) to drill and wring the superposed shellfish sundries absorbed in the pipeline;
9) and taking out the crushed slag from the raw water pipe by using the cleaning device (2) to finish cleaning.
2. The visual robot method for removing shellfish from a raw water pipeline of claim 1, wherein: in the step 1, the binocular vision device (4) firstly calibrates a single eye to obtain an internal parameter matrix and a distortion coefficient matrix of each camera; then binocular calibration is carried out on the binocular vision device (4) to obtain a reprojection matrix for three-dimensional reconstruction and a unit conversion relation between the pixel distance and the millimeter distance; aiming at the brightness inside the road, the LED illumination is adjusted, the optimal illumination is optimized, and the noise caused by illumination is reduced.
3. The visual robot method for removing shellfish from a raw water pipeline of claim 1, wherein: and 2, acquiring a plurality of images of shellfish sundries inside and outside different pipelines by using a visual camera, programming and realizing automatic marking.
4. The visual robot method for removing shellfish from a raw water pipeline of claim 1, wherein: when the mechanical arm of the robot is pushed forwards, the inverted hook can scoop the crushed shellfish and crushed slag with small volume backwards, and then the shellfish and crushed slag are conveyed to the outside through the conveying screw rod.
5. The utility model provides a clear away vision robot device of shellfish in raw water pipeline which characterized in that: the vision robot method is used for realizing the vision robot method of any one of claims 1 to 4, and comprises a robot (1), wherein a cleaning device (2) is arranged on a driving arm of the robot (1), a crushing device (3) and a binocular vision device (4) are respectively arranged at the front end of the cleaning device (2), and the vision robot method further comprises a controller, wherein the binocular vision device (4) is in communication connection with the controller, and the controller is in communication connection with the robot (1), the cleaning device (2) and the crushing device (3) respectively.
6. The visual robot device for removing shellfish from a raw water pipeline of claim 5, wherein: the binocular vision device (4) comprises two vision cameras and a searchlight.
7. The visual robot device for removing shellfish from a raw water pipeline of claim 5, wherein: the cleaning device (2) comprises a motor and a transmission screw rod arranged on the motor.
8. The visual robot device for removing shellfish from a raw water pipeline of claim 7, wherein: the front part of the transmission screw is also provided with a barb for clearing the slag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631864.3A CN113305858B (en) | 2021-06-07 | 2021-06-07 | Visual robot method and device for removing shellfish in raw water pipeline |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631864.3A CN113305858B (en) | 2021-06-07 | 2021-06-07 | Visual robot method and device for removing shellfish in raw water pipeline |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113305858A CN113305858A (en) | 2021-08-27 |
| CN113305858B true CN113305858B (en) | 2022-05-03 |
Family
ID=77377927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110631864.3A Active CN113305858B (en) | 2021-06-07 | 2021-06-07 | Visual robot method and device for removing shellfish in raw water pipeline |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113305858B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113688758B (en) * | 2021-08-31 | 2023-05-30 | 重庆科技学院 | Intelligent recognition system for high-consequence region of gas transmission pipeline based on edge calculation |
| CN117506938B (en) * | 2024-01-04 | 2024-03-26 | 北京隆科兴科技集团股份有限公司 | Method and device for clearing obstacles in pipeline and electronic equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014180720A (en) * | 2013-03-19 | 2014-09-29 | Yaskawa Electric Corp | Robot system and calibration method |
| CN107553497B (en) * | 2017-10-20 | 2023-12-22 | 苏州瑞得恩光能科技有限公司 | Edge positioning device of solar panel cleaning robot and positioning method thereof |
| CN109117701B (en) * | 2018-06-05 | 2022-01-28 | 东南大学 | Pedestrian intention identification method based on graph convolution |
| CN108965982B (en) * | 2018-08-28 | 2020-01-31 | 百度在线网络技术（北京）有限公司 | Video recording method and device, electronic equipment and readable storage medium |
| CN109040693B (en) * | 2018-08-31 | 2020-11-10 | 上海赛特斯信息科技股份有限公司 | Intelligent alarm system and method |
-
2021
- 2021-06-07 CN CN202110631864.3A patent/CN113305858B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113305858A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113305858B (en) | Visual robot method and device for removing shellfish in raw water pipeline | |
| CN108266594B (en) | Automatic inspection dredging device and dredging method for pipeline robot | |
| CN104238566B (en) | Electronic circuit is with inspection robot control system based on image recognition | |
| CN104813975B (en) | Unmanned operation aquaculture robot under water | |
| DE102004021115A1 (en) | Cleaning robot with soil disinfection function | |
| EP3426015A1 (en) | A robotic harvester | |
| WO2011038170A3 (en) | Downhole optical imaging tools and methods | |
| CN205336853U (en) | Intelligence is kept away and is hindered machine people that mows | |
| CN103761565A (en) | Underwater fry, young shrimp and young crab quantity estimating and behavior monitoring device and method based on computer vision | |
| JP2008023630A (en) | Arm-guiding moving body and method for guiding arm | |
| CN104773267A (en) | Self-adaptive fishing ship | |
| DE102016108460A1 (en) | Cleaning robot and method of control | |
| CN114526715B (en) | Old district reforms transform and uses scanning system based on three-dimensional laser scanner | |
| CN116985090A (en) | Intelligent garbage sorting robot | |
| GB2542772A (en) | A device for cleaning a chimney flue | |
| DE102018132964A1 (en) | AUTONOMOUS CLEANING DEVICE WITH A SUCTION ARM | |
| CN114476661B (en) | Automatic water feeding device and method for water dispenser based on three-dimensional point cloud reconstruction | |
| CN216622368U (en) | Protection device based on water quality testing sensor | |
| CN105926556A (en) | Overwater and land two-purpose garbage pick device | |
| KR102111126B1 (en) | Cleaning robots having underwater sludge crusher | |
| CN114617489A (en) | Sweeper for cleaning floor gaps | |
| CN105973308A (en) | Automatic closed chamber detection device based on real-time image acquisition technology and method | |
| CN220133085U (en) | Sewer pipeline dredging robot | |
| CN116098055B (en) | Corn emasculation and weak removal robot and method for emasculation and weak removal by using same | |
| CN206748400U (en) | A kind of industrial carrying machine people based on autonomous learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |