CN115981178B - Simulation system for slaughtering fish and aquatic products - Google Patents
Simulation system for slaughtering fish and aquatic products Download PDFInfo
- Publication number
- CN115981178B CN115981178B CN202211632885.8A CN202211632885A CN115981178B CN 115981178 B CN115981178 B CN 115981178B CN 202211632885 A CN202211632885 A CN 202211632885A CN 115981178 B CN115981178 B CN 115981178B
- Authority
- CN
- China
- Prior art keywords
- actual
- virtual
- mechanical arm
- camera
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 41
- 241000251468 Actinopterygii Species 0.000 title claims abstract description 31
- 238000003307 slaughter Methods 0.000 title claims abstract description 24
- 239000010410 layer Substances 0.000 claims abstract description 23
- 230000000007 visual effect Effects 0.000 claims abstract description 9
- 239000002346 layers by function Substances 0.000 claims abstract description 7
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 5
- 238000005520 cutting process Methods 0.000 claims description 42
- 238000000034 method Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
- Y02A40/81—Aquaculture, e.g. of fish
Landscapes
- Manipulator (AREA)
Abstract
The invention discloses a simulation system for slaughtering fish and aquatic products, which comprises a functional layer, an equipment abstraction layer and an equipment layer; the equipment layer comprises virtual equipment and actual equipment, wherein the virtual equipment comprises a virtual camera, a virtual mechanical arm and a virtual end tool, the actual equipment comprises an actual camera, an actual mechanical arm and an actual end tool, and the virtual equipment and the actual equipment are in state synchronization; the device abstraction layer provides a camera interface for the virtual camera and the actual camera, a mechanical arm interface for the virtual mechanical arm and the actual mechanical arm, and an end tool interface for the virtual end tool and the actual end tool; the functional layer comprises a calibration module, a 3D simulation module and a track planning module. The functional layer, the equipment abstract layer and the equipment layer form a visual application simulation experiment environment in which a visual algorithm and a mechanical arm work cooperatively, and the fish and aquatic products can be killed by a robot according to a track identified by machine vision.
Description
Technical Field
The invention relates to the technical field of industrial robots, in particular to a simulation system for slaughtering fish and aquatic products.
Background
Industrial robots can safely and efficiently complete heavy production tasks and are widely applied to various industries. At present, an industrial robot manual teaching mode can only enable a mechanical arm to walk a fixed route, more requirements on machine vision in actual business are met, and a machine vision detection result is needed to drive the mechanical arm to dynamically adjust a motion track. The application of existing industrial robots is increasingly dependent on machine vision, so perfect cooperation between machine vision and robots is required. The visual algorithm and the mechanical arm can be combined and applied to slaughtering of fish and aquatic products, so that the method is a breakthrough in the prior art.
Disclosure of Invention
The invention aims to provide a simulation system for slaughtering fish and aquatic products.
The technical scheme of the invention is as follows:
a simulation system for slaughtering fish and aquatic products comprises a functional layer, an equipment abstraction layer and an equipment layer;
The equipment layer comprises virtual equipment and actual equipment, wherein the virtual equipment comprises a virtual camera, a virtual mechanical arm and a virtual end tool, the actual equipment comprises an actual camera, an actual mechanical arm and an actual end tool, the virtual camera is in state synchronization with the actual camera, the virtual mechanical arm is in state synchronization with the actual mechanical arm, and the virtual end tool is in state synchronization with the actual end tool;
The device abstraction layer provides a camera interface for the virtual camera and the actual camera, provides a mechanical arm interface for the virtual mechanical arm and the actual mechanical arm, and provides an end tool interface for the virtual end tool and the actual end tool so as to uniformly manage the virtual device and the actual device and support the state synchronization of the simulation environment and the actual environment;
the function layer comprises a calibration module, a 3D simulation module and a track planning module, wherein the calibration module provides the position calibration functions of an actual camera, an actual mechanical arm and an actual end tool, the 3D simulation module provides the simulation display of a simulation environment, the track planning module provides the calculation function of a cutting track, and the cutting track is calculated according to the scanning point cloud result of the actual camera.
Wherein the actual camera is a 3D camera.
The actual mechanical arm is a six-axis mechanical arm.
Wherein the actual end tool is an electrode cutter.
The fish aquatic product slaughtering method of the simulation system for slaughtering fish aquatic products comprises the following steps:
S1, starting a virtual device and an actual device at the same time;
S2, loading and displaying virtual equipment by simulation software, and connecting actual equipment;
s3, calibrating positions of an actual camera, an actual mechanical arm and an actual end tool;
s4, synchronizing the states of the virtual equipment and the actual equipment, and simulating a real environment in the virtual environment;
S5, fixing the fish at the cutting position;
s6, scanning the point cloud by an actual camera, and synchronously displaying the point cloud in a virtual simulation scene;
s7, calculating a cutting track and displaying the track in simulation software;
s8, converting the cutting track into an actual mechanical arm moving track;
s9, simulating a cutting process in simulation software;
s10, performing a real cutting experiment in an actual environment.
Further, in step S6, 2D image data and 3D point cloud data are obtained by photographing with an actual camera.
Further, the method for calculating the cutting track in step S7 includes:
s71, identifying four cutting points of the back of the fish by using the 2D image data and using AI;
s72, fitting an ellipse by using four cutting points to form a 2D cutting track;
S73, converting the 2D cutting track into a 3D cutting track by using the 3D point cloud data.
Further, step S8, after determining the 3D cutting track, calculates the movement track of the center point of the flange at the end of the actual mechanical arm according to the calibrated tool coordinates.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, a visual algorithm and a visual application simulation experiment environment for the cooperation of the mechanical arm are formed by the calibration module, the 3D simulation module, the track planning module, the equipment abstraction layer, the virtual camera, the actual camera, the virtual mechanical arm, the actual mechanical arm, the virtual end tool and the actual end tool, and the slaughtering of fish and aquatic products can be realized by the robot according to the track identified by machine vision.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a system diagram of a simulation system for slaughtering fish and aquatic products;
Fig. 2 is a flow chart of a simulation method for slaughtering fish and aquatic products.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In order to illustrate the technical scheme of the invention, the following description is made by specific examples.
Examples
Referring to fig. 1, the embodiment provides a simulation system for slaughtering fish and aquatic products, which comprises a functional layer, an equipment abstraction layer and an equipment layer.
The equipment layer includes virtual equipment and actual equipment, and virtual equipment includes virtual camera, virtual arm and virtual end instrument, and actual equipment includes actual camera, actual arm and actual end instrument, and in this embodiment, actual camera is 3D camera, and actual arm is six arms, and actual end instrument is electrode cutterbar, and virtual camera is state synchronization with actual camera, and virtual arm is state synchronization with actual arm, and virtual end instrument is state synchronization with actual end instrument.
The device abstraction layer provides a camera interface for the virtual camera and the actual camera, a mechanical arm interface for the virtual mechanical arm and the actual mechanical arm, and an end tool interface for the virtual end tool and the actual end tool, so as to uniformly manage the virtual device and the actual device and support the state synchronization of the simulation environment and the actual environment.
The functional layer comprises a calibration module, a 3D simulation module and a track planning module, wherein the calibration module provides the position calibration functions of an actual camera, an actual mechanical arm and an actual end tool, the 3D simulation module provides the simulation display of a simulation environment, the track planning module provides the calculation function of a cutting track, and the cutting track is calculated according to the scanning point cloud result of the actual camera.
The visual application simulation experiment environment of the cooperation work of the visual algorithm and the mechanical arm is formed by the calibration module, the 3D simulation module, the track planning module, the equipment abstraction layer, the virtual camera, the actual camera, the virtual mechanical arm, the actual mechanical arm, the virtual end tool and the actual end tool, and the slaughtering of fish and aquatic products can be realized by the robot according to the track identified by the machine vision.
The fish aquatic product slaughtering method of the simulation system for slaughtering fish aquatic products comprises the following steps as shown in fig. 2:
S1, starting a virtual device and an actual device at the same time;
S2, loading and displaying virtual equipment by simulation software, and connecting actual equipment;
s3, calibrating positions of an actual camera, an actual mechanical arm and an actual end tool;
once the positional relationship is fixed, the calibration process is only needed once;
s4, synchronizing the states of the virtual equipment and the actual equipment, and simulating a real environment in the virtual environment;
S5, fixing the fish at the cutting position;
s6, scanning the point cloud by an actual camera, and synchronously displaying the point cloud in a virtual simulation scene;
Specifically, taking a picture through an actual camera to obtain 2D image data and 3D point cloud data;
s7, calculating a cutting track and displaying the track in simulation software;
The cutting track calculating method comprises the following steps:
s71, identifying four cutting points of the back of the fish by using the 2D image data and using AI;
s72, fitting an ellipse by using four cutting points to form a 2D cutting track;
s73, converting the 2D cutting track into a 3D cutting track by utilizing the 3D point cloud data;
s8, converting the cutting track into an actual mechanical arm moving track;
specifically, after the 3D cutting track is determined, calculating the movement track of the center point of the flange plate at the tail end of the actual mechanical arm according to the calibrated tool coordinates.
S9, simulating a cutting process in simulation software;
s10, performing a real cutting experiment in an actual environment.
The foregoing description of the preferred embodiment of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (7)
1. A simulation system for slaughtering fish and aquatic products, which is characterized in that: the device comprises a functional layer, a device abstraction layer and a device layer;
The equipment layer comprises virtual equipment and actual equipment, wherein the virtual equipment comprises a virtual camera, a virtual mechanical arm and a virtual end tool, the actual equipment comprises an actual camera, an actual mechanical arm and an actual end tool, the virtual camera is in state synchronization with the actual camera, the virtual mechanical arm is in state synchronization with the actual mechanical arm, and the virtual end tool is in state synchronization with the actual end tool;
The device abstraction layer provides a camera interface for the virtual camera and the actual camera, provides a mechanical arm interface for the virtual mechanical arm and the actual mechanical arm, and provides an end tool interface for the virtual end tool and the actual end tool so as to uniformly manage the virtual device and the actual device and support the state synchronization of the simulation environment and the actual environment;
The function layer comprises a calibration module, a 3D simulation module and a track planning module, wherein the calibration module provides the position calibration functions of an actual camera, an actual mechanical arm and an actual end tool, the 3D simulation module provides the simulation display of a simulation environment, the track planning module provides the calculation function of a cutting track, and the cutting track is calculated according to the scanning point cloud result of the actual camera;
The visual application simulation experiment environment in which a visual algorithm and a mechanical arm work cooperatively is formed by a calibration module, a 3D simulation module, a track planning module, an equipment abstraction layer, a virtual camera, an actual camera, a virtual mechanical arm, an actual mechanical arm, a virtual end tool and an actual end tool can realize slaughtering of fish and aquatic products according to a track identified by machine vision through a robot, and the specific method comprises the following steps:
S1, starting a virtual device and an actual device at the same time;
S2, loading and displaying virtual equipment by simulation software, and connecting actual equipment;
s3, calibrating positions of an actual camera, an actual mechanical arm and an actual end tool;
s4, synchronizing the states of the virtual equipment and the actual equipment, and simulating a real environment in the virtual environment;
S5, fixing the fish at the cutting position;
s6, scanning the point cloud by an actual camera, and synchronously displaying the point cloud in a virtual simulation scene;
s7, calculating a cutting track and displaying the track in simulation software;
s8, converting the cutting track into an actual mechanical arm moving track;
s9, simulating a cutting process in simulation software;
s10, performing a real cutting experiment in an actual environment.
2. A simulation system for slaughtering fish and aquatic products according to claim 1, characterised in that: the actual camera is a 3D camera.
3. A simulation system for slaughtering fish and aquatic products according to claim 1, characterised in that: the actual mechanical arm is a six-axis mechanical arm.
4. A simulation system for slaughtering fish and aquatic products according to claim 1, characterised in that: the actual end tool is an electrode cutter.
5. A simulation system for slaughtering fish and aquatic products according to claim 1, characterized by step S6, obtaining 2D image data and 3D point cloud data by photographing with an actual camera.
6. A simulation system for slaughtering fish and aquatic products according to claim 5, wherein the method for calculating the cutting trajectory in step S7 comprises:
s71, identifying four cutting points of the back of the fish by using the 2D image data and using AI;
s72, fitting an ellipse by using four cutting points to form a 2D cutting track;
S73, converting the 2D cutting track into a 3D cutting track by using the 3D point cloud data.
7. The simulation system for slaughtering fish and aquatic products according to claim 6, wherein in step S8, after determining the 3D cutting trajectory, the movement trajectory of the center point of the flange at the end of the actual mechanical arm is calculated according to the calibrated tool coordinates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211632885.8A CN115981178B (en) | 2022-12-19 | 2022-12-19 | Simulation system for slaughtering fish and aquatic products |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211632885.8A CN115981178B (en) | 2022-12-19 | 2022-12-19 | Simulation system for slaughtering fish and aquatic products |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115981178A CN115981178A (en) | 2023-04-18 |
| CN115981178B true CN115981178B (en) | 2024-05-24 |
Family
ID=85975219
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211632885.8A Active CN115981178B (en) | 2022-12-19 | 2022-12-19 | Simulation system for slaughtering fish and aquatic products |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981178B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101004604A (en) * | 2006-01-18 | 2007-07-25 | 中国科学院自动化研究所 | Cooperation control system for multi-bionic robot |
| CN104067781A (en) * | 2014-06-16 | 2014-10-01 | 华南农业大学 | Virtual robot and real robot integration based picking system and method |
| WO2018072730A1 (en) * | 2016-10-21 | 2018-04-26 | 遨博（北京）智能科技有限公司 | Robot simulation control method and apparatus |
| CN110308797A (en) * | 2019-07-09 | 2019-10-08 | 西北工业大学 | Underwater robot environmental interaction system based on body-sensing technology mechanical arm and virtual reality technology |
| CN111452034A (en) * | 2019-01-21 | 2020-07-28 | 广东若铂智能机器人有限公司 | Double-camera machine vision intelligent industrial robot control system and control method |
| WO2021085727A1 (en) * | 2019-10-28 | 2021-05-06 | 주식회사 엠앤디 | Cutting robot system and simulation method therefor |
| EP3970925A1 (en) * | 2019-05-17 | 2022-03-23 | Kabushiki Kaisha Yaskawa Denki | Robot system, recovery program generation device, control support device, control device, program, recovery program generation method, and recovery program output method |
| CN114789450A (en) * | 2022-06-02 | 2022-07-26 | 深慧视(深圳)科技有限公司 | Robot motion trajectory digital twinning method based on machine vision |
| CN115154281A (en) * | 2022-07-28 | 2022-10-11 | 广东若铂智能机器人有限公司 | Massage method of massage robot |
| CN217695120U (en) * | 2022-01-27 | 2022-11-01 | 西安理工大学 | Accurate positioning type fish killing and scale removing mechanism |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014176334A1 (en) * | 2013-04-23 | 2014-10-30 | Northwestern University | Translational parallel manipulators and methods of operating the same |
| US20220382246A1 (en) * | 2021-04-28 | 2022-12-01 | Nvidia Corporation | Differentiable simulator for robotic cutting |
-
2022
- 2022-12-19 CN CN202211632885.8A patent/CN115981178B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101004604A (en) * | 2006-01-18 | 2007-07-25 | 中国科学院自动化研究所 | Cooperation control system for multi-bionic robot |
| CN104067781A (en) * | 2014-06-16 | 2014-10-01 | 华南农业大学 | Virtual robot and real robot integration based picking system and method |
| WO2018072730A1 (en) * | 2016-10-21 | 2018-04-26 | 遨博（北京）智能科技有限公司 | Robot simulation control method and apparatus |
| CN111452034A (en) * | 2019-01-21 | 2020-07-28 | 广东若铂智能机器人有限公司 | Double-camera machine vision intelligent industrial robot control system and control method |
| EP3970925A1 (en) * | 2019-05-17 | 2022-03-23 | Kabushiki Kaisha Yaskawa Denki | Robot system, recovery program generation device, control support device, control device, program, recovery program generation method, and recovery program output method |
| CN110308797A (en) * | 2019-07-09 | 2019-10-08 | 西北工业大学 | Underwater robot environmental interaction system based on body-sensing technology mechanical arm and virtual reality technology |
| WO2021085727A1 (en) * | 2019-10-28 | 2021-05-06 | 주식회사 엠앤디 | Cutting robot system and simulation method therefor |
| CN217695120U (en) * | 2022-01-27 | 2022-11-01 | 西安理工大学 | Accurate positioning type fish killing and scale removing mechanism |
| CN114789450A (en) * | 2022-06-02 | 2022-07-26 | 深慧视(深圳)科技有限公司 | Robot motion trajectory digital twinning method based on machine vision |
| CN115154281A (en) * | 2022-07-28 | 2022-10-11 | 广东若铂智能机器人有限公司 | Massage method of massage robot |
Non-Patent Citations (2)
| Title |
|---|
| Design and Simulation of a Vision-Based Automatic Trout Fish-Processing Robot;Hossein Azarmdel 等;《applied sciences》;20210617;1-30 * |
| 基于运动描述语言的机械臂轨迹生成及仿真;刘钊铭 等;《计算机仿真》;20190630;第36卷(第6期);310-315 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115981178A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7334239B2 (en) | Robot Calibration for Augmented Reality and Digital Twins | |
| CN108958233B (en) | Perception simulation method and device | |
| CN112634318B (en) | Teleoperation system and method for underwater maintenance robot | |
| CN110142770B (en) | Robot teaching system and method based on head-mounted display device | |
| JP2005074600A (en) | Robot and robot moving method | |
| US20190321983A1 (en) | Robot movement teaching apparatus, robot system, and robot controller | |
| WO2022000713A1 (en) | Augmented reality self-positioning method based on aviation assembly | |
| CN114912287A (en) | Robot autonomous grabbing simulation system and method based on target 6D pose estimation | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN112847336B (en) | Action learning method and device, storage medium and electronic equipment | |
| CN114347033A (en) | Robot article grabbing method and device, robot and storage medium | |
| CN111476909A (en) | Teleoperation control method and teleoperation control system for compensating time delay based on virtual reality | |
| CN114299039B (en) | Robot and collision detection device and method thereof | |
| CN117103277A (en) | Mechanical arm sensing method based on multi-mode data fusion | |
| CN210361314U (en) | Robot teaching device based on augmented reality technology | |
| CN115981178B (en) | Simulation system for slaughtering fish and aquatic products | |
| CN115514885B (en) | Remote augmented reality follow-up sensing system and method based on monocular and binocular fusion | |
| CN116175582A (en) | Intelligent mechanical arm control system and control method based on machine vision | |
| CN113920191B (en) | 6D data set construction method based on depth camera | |
| CN114029940B (en) | Motion path planning method, device, equipment, medium and mechanical arm | |
| Cho et al. | Development of VR visualization system including deep learning architecture for improving teleoperability | |
| US11752632B2 (en) | Actuated mechanical machine calibration to stationary marker | |
| CN211890823U (en) | Four-degree-of-freedom mechanical arm vision servo control system based on RealSense camera | |
| JP2837393B2 (en) | Object processing method and apparatus | |
| CN114029950A (en) | Robot coordinate system analysis method and device, robot device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |