CN114396889B - Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof - Google Patents
Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof Download PDFInfo
- Publication number
- CN114396889B CN114396889B CN202210031065.7A CN202210031065A CN114396889B CN 114396889 B CN114396889 B CN 114396889B CN 202210031065 A CN202210031065 A CN 202210031065A CN 114396889 B CN114396889 B CN 114396889B
- Authority
- CN
- China
- Prior art keywords
- butter
- turtle shell
- hooking
- cutting
- tail end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000014121 butter Nutrition 0.000 title claims abstract description 143
- 241000270666 Testudines Species 0.000 title claims abstract description 93
- 210000001015 abdomen Anatomy 0.000 title claims abstract description 58
- 230000000007 visual effect Effects 0.000 title claims abstract description 37
- 238000012545 processing Methods 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 11
- 238000005520 cutting process Methods 0.000 claims abstract description 71
- 230000003187 abdominal effect Effects 0.000 claims description 27
- 238000004519 manufacturing process Methods 0.000 claims description 22
- 230000033001 locomotion Effects 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 12
- 239000011664 nicotinic acid Substances 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims description 6
- 230000009471 action Effects 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 7
- 241001482311 Trionychidae Species 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 230000004075 alteration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 210000000078 claw Anatomy 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 235000019197 fats Nutrition 0.000 description 2
- 238000003307 slaughter Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 235000016709 nutrition Nutrition 0.000 description 1
- 230000035764 nutrition Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 235000021067 refined food Nutrition 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/2433—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures for measuring outlines by shadow casting
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Manipulator (AREA)
- Processing Or Creating Images (AREA)
Abstract
A visual positioning and hooking system for automatically processing line turtle shell intra-abdominal butter and a control method thereof comprise a five-degree-of-freedom robot; the five-degree-of-freedom robot comprises a ball screw axial moving device, a ball screw transverse moving device, a worm gear device, a steering engine and a tail end hooking and cutting actuator; the tail end hooking and cutting actuator comprises a ZED binocular camera, a cutter finger fixing seat and a cutter finger, and the tail end hooking and cutting actuator controls the cutter finger to open and close through a stepping motor so as to cut butter in the abdomen of the turtle shell; the tail end hooking and cutting actuator realizes 360-degree rotation by means of a steering engine; the ball screw axial moving device is used for controlling the tail end hooking and cutting actuator to move along the vertical direction; the ball screw transverse moving device is used for controlling the ball screw axial moving device and the tail end hooking and cutting actuator to transversely move; the automatic hook-cutting turtle shell intra-abdominal butter replaces manual butter cutting, visual detection of butter by a binocular camera is used for positioning butter cutting and then collecting, and the cutting and collecting efficiency is high and pollution-free.
Description
Technical Field
The invention relates to the field of visual detection and machinery of automatic processing production lines, in particular to a visual positioning and hooking system of butter in the abdomen of turtle shells of an automatic processing line and a control method thereof.
Background
The turtle shell is rich in nutrition and has high medicinal value. At present, large-scale cultivation is carried out, and automatic processing becomes a necessary trend, particularly epidemic outbreaks worldwide, so that some processed foods and packages thereof are infected. Therefore, the pollution-free slaughter can be realized only by realizing the intellectualization and the automation, and the machine vision of the artificial intelligence is a key technology for realizing the unmanned, intelligent and automation. Butter is fat of the abdomen of the soft-shelled turtle close to four legs, has high medical value, is particularly fishy, and sometimes the fat is partially shielded from the inside, so that the soft-shelled turtle needs to be automatically resected in the slaughtering process. Two technical problems to be solved in realizing automation and intellectualization are: firstly, the soft-shelled turtle belly butter is identified and positioned through vision and is controlled by vision servo, and secondly, the automatic hooking and cutting device is designed to be light and handy.
According to patent search and market research, related prior art disclosure is not found, so that a visual positioning and hooking system for automatically processing the butter in the turtle shell abdomen and a control method thereof are developed.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and aims to overcome the high cost and pollution risk caused by manual processing and realize automatic cutting. Firstly, a visual platform is creatively put forward, a butter part is identified and positioned, butter pixel coordinates are segmented by using a semantic segmentation model based on a full convolution neural network, then a disparity map is obtained through stereo matching, further three-dimensional point cloud information of butter is extracted, the purpose of detection and positioning is achieved, and after the three-dimensional coordinates of visual positioning are obtained, a visual servo control executing mechanism and a cutter thereof are realized to complete a work task. Then innovatively designing a flexible bionic multi-blade cutter part with the expandable diameter, wherein the cutter edge is similar to a human nail, the cutter body can be bent in two degrees of freedom, and when butter is cut and clamped by the cutter fingers, the cutter bar is upwards, and at the moment, the butter is cut out by a hook and placed in a container.
In order to achieve the aim, the technical scheme adopted by the invention is to provide a visual positioning and hooking system for butter in the abdomen of the turtle shell of an automatic processing line, which is characterized by comprising a five-degree-of-freedom robot; the five-degree-of-freedom robot comprises a ball screw axial moving device, a ball screw transverse moving device, a worm gear device, a steering engine and a tail end hooking and cutting actuator;
the tail end hooking and cutting actuator comprises a ZED binocular camera, a cutter finger fixing seat and a cutter finger, and the tail end hooking and cutting actuator controls the cutter finger to open and close through a stepping motor so as to cut butter in the abdomen of the turtle shell; the tail end hooking and cutting actuator realizes 360-degree rotation by means of the steering engine;
the ball screw axial moving device is used for controlling the tail end hooking and cutting actuator to move along the vertical direction; the ball screw transverse moving device is used for controlling the ball screw axial moving device and the tail end hooking and cutting actuator to transversely move;
the five-degree-of-freedom robot is connected to the robot base, the ZED binocular camera is connected with the input end of the controller, so that visual information of the butter in the turtle shell abdomen on the automatic processing production line captured by the ZED binocular camera is transmitted to the controller through a network, and the controller is connected with the execution module, so that the tail end hooking and cutting executor is instructed to reach the butter position to hook and cut and collect the butter in the turtle shell abdomen.
In one or more embodiments of the present invention, the visual information data collected by the ZED binocular camera includes three-dimensional spatial position data of butter, RGB values, and depth values.
In one or more embodiments of the present invention, the ZED binocular camera is fixedly mounted on the cutter finger fixing base; one end of the knife finger fixing seat is fixed on the steering engine flange, and the other end of the knife finger fixing seat is connected with a knife finger; the knife finger controls the opening and closing size by means of the forward and reverse rotation of the stepping motor.
In one or more embodiments of the present invention, the five degree of freedom robot is installed on a turtle shell automated processing line; the worm gear device, the ball screw transverse moving device and the ball screw axial moving device are respectively provided with motors for respective control and independent movement; the controller is used for storing and controlling the motion information data of the five-degree-of-freedom robot.
In one or more embodiments of the present invention, the end hooking and cutting actuator is a four-jaw manipulator, the fingers are welded with flexible bionic multi-blade cutters with expandable diameters, the width of the flexible bionic multi-blade cutters is set to be 20-30 mm, the number of the fingers is four, and every two adjacent fingers are separated by 90 degrees and are uniformly arranged on the finger fixing seat.
The invention also provides a control method of the visual positioning and hooking system of the butter in the turtle shell abdomen of the automatic processing line, which comprises the following control steps of:
calibrating a ZED binocular camera to obtain an internal parameter matrix, a distortion coefficient and a re-projection matrix data of the camera, thereby correcting left and right views and converting coordinates;
step two, performing hand-eye calibration on the five-degree-of-freedom robot and the ZED binocular camera to obtain a camera coordinate system and a rotation matrix and a translation matrix of the robot coordinate system;
step three, collecting a large number of turtle shell abdominal butter RGB images in advance, and constructing a turtle shell abdominal butter RGB image sample library;
step four, manually labeling the turtle shell abdominal butter in the image by using an image labeling tool Labelme, manufacturing a turtle shell abdominal butter RGB image data set, and converting the RGB image data set into a Tf record format file;
fifthly, the turtle shell abdominal butter RGB image data set is processed according to the following steps of 6:2:2 is divided into a training set, a verification set and a test set;
training by using a semantic segmentation model deep V3 based on a full convolution neural network to obtain a semantic segmentation model of turtle shell abdominal butter;
fixing the five-degree-of-freedom robot on an automatic processing production line, and adjusting the posture to enable the ZED binocular camera to be opposite to the automatic processing production line;
step eight, opening a ZED binocular camera to acquire an RGB image of the turtle shell belly in real time, calling a pre-trained semantic segmentation model of the turtle shell belly butter to segment the turtle shell belly butter area in real time, and carrying out local stereo matching to calculate the cloud information of the turtle shell belly Huang Youdian;
step nine, calculating the center position coordinates of the turtle shell abdomen butter according to the cloud information of the turtle shell abdomen Huang Youdian;
step ten, controlling the tail end hooking and cutting actuator to move to the position right above the turtle shell abdominal butter according to the calculated center position coordinates of the turtle shell abdominal butter, and then descending and opening the tail end hooking and cutting actuator to finish hooking and cutting actions;
step eleven, after hooking, the tail end hooking actuator ascends and returns to the upper part of the butter collecting box, descends and opens, and the turtle shell abdominal butter is placed into the butter collecting box;
and step twelve, turning the robot back to the upper part of the production line, starting the next round of visual recognition to the butter position in the other turtle shell abdomen, and repeatedly and circularly carrying out the steps until the automatic processing production line stops butter cutting and collecting work.
Compared with the background technology, the invention has the following beneficial effects:
due to the adoption of the scheme, the five-degree-of-freedom robot, the tail end hooking and cutting actuator and the cutter fingers are adopted, the butter in the abdomen of the automatic hooking and cutting turtle shell can be used for manually cutting the butter, the visual detection of the butter by the binocular camera is used for positioning the collection after the butter is cut, and the cutting and collecting efficiency is high and pollution is avoided.
The invention innovatively provides to build a visual platform, a semantic segmentation model based on a full convolution neural network is used for segmenting butter pixel coordinates, then a disparity map is obtained through stereo matching, further three-dimensional point cloud information of butter is extracted, and butter parts are identified and positioned, so that the purposes of detection and positioning are achieved.
According to the invention, the ZED binocular camera is adopted to detect and position the butter visual image, the deep labV3 network is used for training the shot butter image, so that the butter can be accurately identified and positioned, then the knife finger is controlled to realize high-precision cutting and collecting of the butter, the butter in the turtle shell abdomen can be completely cut off, and the integrity of the whole butter is ensured.
The invention adopts the five-degree-of-freedom robot, has higher degree of freedom, can realize the higher-efficiency cutting and collecting of butter on an automatic processing production line, has better sensitivity, and is suitable for various growth postures of butter in the abdomen of the turtle shells in different states.
Drawings
FIG. 1 is a schematic diagram of a visual positioning and hooking system for butter in the abdomen of turtle shell of an automatic processing line according to an embodiment of the present invention;
FIG. 2 is a schematic view showing a visual positioning and hooking system for butter in the abdomen of a turtle shell of an automatic processing line installed on a production line according to an embodiment of the present invention;
FIG. 3 is a schematic view of a visual positioning and hooking actuator for butter in the abdomen of a turtle shell of an automatic processing line according to an embodiment of the present invention;
FIG. 4 is a block diagram of an automatic launch control procedure for visually locating a water surface target in accordance with an embodiment of the present invention;
the shapes, configurations and understanding of the resulting arrangements are illustrated in the drawings, the various components of the drawings are not necessarily to scale, and the dimensions of the various components and elements of the drawings may be exaggerated or reduced to more clearly illustrate the embodiments of the present invention described herein.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout.
The orientations shown in the drawings are not to be construed as limiting the specific scope of the invention, and only as a reference to the preferred embodiments, variations in the positions or numbers of product components shown in the drawings or structural simplifications may be made.
The terms "connected" and "connected" as used in the specification and illustrated in the drawings refer to the components as being "connected" to each other, and are understood to mean fixedly connected or detachably connected or integrally connected; the connection can be direct connection or connection through an intermediate medium, and a person skilled in the art can understand the connection relation according to specific situations to obtain a screw connection or riveting or welding or clamping or embedding and other modes to replace the modes in different embodiments in a proper mode.
Terms of orientation such as up, down, left, right, top, bottom, and the like, as well as orientations shown in the drawings, may be used for direct contact or contact by additional features between the components; such as directly above and obliquely above, or it merely represents above the other; other orientations may be understood by analogy.
The technical scheme and the beneficial effects of the invention are more clear and definite by further describing the specific embodiments of the invention with reference to the accompanying drawings of the specification; and are intended to be illustrative of the invention and are not to be construed as limiting the invention.
Referring to fig. 1-3, a preferred embodiment of the present invention is a visual positioning and hooking system for butter in the abdomen of an automated processing line turtle shell, comprising a five degree of freedom robot 1; the five-degree-of-freedom robot 1 comprises a ball screw axial moving device 11, a ball screw transverse moving device 12, a worm gear device 13, a steering engine 14 and a tail end hooking and cutting actuator 15; the tail end hooking and cutting actuator 15 comprises a ZED binocular camera 151, a cutter finger fixing seat 152 and cutter fingers 153, and the tail end hooking and cutting actuator 153 controls the cutter fingers 153 to open and close through a stepping motor 154 so as to cut butter in the abdomen of the turtle shell; the end hooking actuator 15 is rotated by 360 ° by means of the steering gear 14.
The ball screw axial moving device 11 is used for controlling the tail end hooking and cutting actuator 15 to move up and down along the vertical direction; the ball screw lateral movement device 12 is used for controlling the ball screw axial movement device 11 and the tail end hooking and cutting actuator 15 to move laterally.
The five-degree-of-freedom robot 1 is connected to the robot base 2; the ZED binocular camera 151 is connected with an input end of a controller (not shown in the figure) so that visual information of the butter in the turtle shell abdomen on the automatic processing production line captured by the ZED binocular camera 151 is transmitted to the controller through a network, and the controller is connected with an execution module; thereby instructing the end hooking and cutting actuator 15 to reach the butter position for hooking and collecting butter in the abdomen of the turtle shell; visual information data collected by the ZED binocular camera comprises three-dimensional space position data, RGB values and depth values of butter.
The ZED binocular camera 151 is fixedly arranged on the cutter finger fixing seat 152; one end of a knife finger 153 fixing seat is fixed on the steering engine flange, and the other end of the knife finger fixing seat 152 is connected with a knife finger 153; the opening and closing of the knife finger 153 is controlled by means of the forward and backward rotation of the stepping motor.
The five-degree-of-freedom robot 1 is arranged on a turtle shell automatic processing production line 3; the worm gear device 13, the ball screw transverse moving device 12 and the ball screw axial moving device 11 are respectively provided with motors for respective control and independent movement; the controller is used for storing and controlling the motion information data of the five-degree-of-freedom robot.
The end hooking and cutting actuator 15 may be a four-jaw manipulator, the cutter fingers 153 are welded with flexible bionic multi-blade cutters 155 with expandable diameters, the width of each flexible bionic multi-blade cutter 155 is set to be 20-30 mm, the number of the cutter fingers 153 is four, and every two adjacent cutter fingers are separated by 90 degrees and are uniformly arranged on the cutter finger fixing seat 152. The knife finger 153 is movably connected with a transmission link 156.
The invention further provides a control method of the visual positioning and hooking system of the butter in the abdomen of the turtle shell of the automatic processing line, which comprises the following control steps of:
calibrating a ZED binocular camera to obtain an internal parameter matrix, a distortion coefficient and a re-projection matrix data of the camera, thereby correcting left and right views and converting coordinates;
step two, performing hand-eye calibration on the five-degree-of-freedom robot and the ZED binocular camera to obtain a camera coordinate system and a rotation matrix and a translation matrix of the robot coordinate system;
step three, collecting a large number of turtle shell abdominal butter RGB images in advance, and constructing a turtle shell abdominal butter RGB image sample library;
step four, manually labeling the turtle shell abdominal butter in the image by using an image labeling tool Labelme, manufacturing a turtle shell abdominal butter RGB image data set, and converting the RGB image data set into a Tf record format file;
fifthly, the turtle shell abdominal butter RGB image data set is processed according to the following steps of 6:2:2 is divided into a training set, a verification set and a test set;
training by using a semantic segmentation model deep V3 based on a full convolution neural network to obtain a semantic segmentation model of turtle shell abdominal butter;
fixing the five-degree-of-freedom robot on an automatic processing production line, and adjusting the posture to enable the ZED binocular camera to be opposite to the automatic processing production line;
step eight, opening a ZED binocular camera to acquire an RGB image of the turtle shell belly in real time, calling a pre-trained semantic segmentation model of the turtle shell belly butter to segment the turtle shell belly butter area in real time, and carrying out local stereo matching to calculate the cloud information of the turtle shell belly Huang Youdian;
step nine, calculating the center position coordinates of the turtle shell abdomen butter according to the cloud information of the turtle shell abdomen Huang Youdian;
step ten, controlling the tail end hooking and cutting actuator to move to the position right above the turtle shell abdominal butter according to the calculated center position coordinates of the turtle shell abdominal butter, and then descending and opening the tail end hooking and cutting actuator to finish hooking and cutting actions;
step eleven, after hooking, the tail end hooking actuator ascends and returns to the upper part of the butter collecting box, descends and opens, and the turtle shell abdominal butter is placed into the butter collecting box;
and step twelve, turning the robot back to the upper part of the production line, starting the next round of visual recognition to the butter position in the other turtle shell abdomen, and repeatedly and circularly carrying out the steps until the automatic processing production line stops butter cutting and collecting work.
When in use, the knife finger drives the transmission connecting rod 156 through the forward and reverse rotation of the stepping motor, thereby driving the knife finger 153 to hook and cut the butter in the turtle shell abdomen; the hooking and cutting work of butter with different diameters is realized under the drive of a stepping motor; the size of the butter in the abdomen of the turtle shell is affected by the age of the turtle shell, and the butter in the abdomen of the turtle shell which is normally fed artificially is generally 15-25 mm, so that the width of the flexible bionic multi-blade cutter with the expandable diameter welded on each cutter finger 153 is set to 20-30 mm, and the butter is prevented from falling in the hooking and cutting process. Meanwhile, the flexible bionic multi-blade cutters 155 at the front end of each cutter finger 153 can be designed to be similar to the shape of a human nail so as to facilitate the hooking and taking off of butter. The four cutter fingers 153 are uniformly arranged on the cutter finger fixing seat 152 every 90 degrees apart.
The hook cutting and opening of the cutter finger 153 are controlled by a stepping motor, the cutter finger 153 is connected to a stepping motor fixing plate through a transmission connecting rod 153, when receiving a control instruction, the worm gear and worm device 13 rotates, the ball screw transverse moving device 12 and the ball screw axial moving device 11 cooperatively move to the upper part of the butter position in the turtle shell, the stepping motor rotates positively, a flange arranged on a shaft of the stepping motor 154 is connected with the flange fixing plate to drive the connecting rod to move downwards, so that the four cutter fingers start to be closed and vertically inserted into the butter position.
The stepper motor 154 continues to rotate forward, and the lower ends of the cutter fingers clamp the butter; after cutting, the whole end hooking and cutting actuator 15 rotates for one circle, and the stepping motor continues to rotate forward to reduce the space between four claws and lift butter at the same time; when the cutting is completed, the stepping motor 154 reverses to restore the original state of the mechanical claw, so that the cutter fingers 153 are opened, and the opening, closing and cutting functions of the tail end hooking and cutting actuator 15 are completed, so that the hooking and cutting work of butter is realized. The degree of closure hooking of the knife finger 153 can be varied by varying the steering of the stepper motor.
When the butter cutting machine is operated, after a command of cutting butter, the controller controls the robot to reach the position of the butter, then the stepping motor rotates forward, so that the knife fingers slowly start to be closed until the butter is completely cut, at the moment, the butter is clamped, then the tail end hooking and cutting actuator 5 rotates for 360 degrees again and then continues to reduce the distance between the blades, and finally the robot moves upwards by a set distance, so that the butter is separated from the turtle shell legs; after the butter is completely separated from the turtle shell legs, the tail end hooking and cutting actuator automatically moves to the upper part of the butter collecting box 4 and releases the butter, so that the purpose of hooking and cutting and collecting the butter is achieved.
The worm gear means 13, the ball screw lateral movement means 12 and the ball screw axial movement means 11 of the five degree of freedom robot are all equipped with a specially used motor to control the respective independent movements. The steering action of the two shafts can be adjusted through the worm gear device 13, and the connected ball screw transverse moving device 13 and the ball screw axial moving device 11 can flexibly and horizontally and vertically move to the upper part of the butter position. The controller is used for storing and controlling the motion information data of the robot. Therefore, the automatic hooking and cutting of butter in the turtle shell abdomen is realized to replace manual butter cutting, visual detection of butter by a binocular camera is used for positioning the collection after butter cutting, and the cutting and collecting efficiency and the pollution-free degree are high.
The invention innovatively provides to build a visual platform, a semantic segmentation model based on a full convolution neural network is used for segmenting butter pixel coordinates, then a disparity map is obtained through stereo matching, further three-dimensional point cloud information of butter is extracted, and butter parts are identified and positioned, so that the purposes of detection and positioning are achieved. The ZED binocular camera is adopted to detect and position butter visual images, the deep V3 network is used for training the shot butter images, accurate identification and positioning of butter can be achieved, then the knife finger is controlled to achieve high-precision cutting and collecting of butter, the butter in the turtle shell abdomen can be completely cut down, and the integrity of the whole butter is guaranteed.
While this invention has been described in terms of the preferred embodiment as described above, there are alterations, permutations, and equivalents which fall within the scope of this invention; numerous alternatives are possible to implement the invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention; it will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described above, but that modifications and substitutions using techniques known in the art on the basis of the present invention fall within the scope of the present invention, which is defined by the claims.
Claims (6)
1. A visual positioning and hooking system for automatically processing butter in the abdomen of a line turtle shell is characterized by comprising a five-degree-of-freedom robot; the five-degree-of-freedom robot comprises a ball screw axial moving device, a ball screw transverse moving device, a worm gear device, a steering engine and a tail end hooking and cutting actuator;
the tail end hooking and cutting actuator comprises a ZED binocular camera, a cutter finger fixing seat and a cutter finger, and the tail end hooking and cutting actuator controls the cutter finger to open and close through a stepping motor so as to cut butter in the abdomen of the turtle shell; the tail end hooking and cutting actuator realizes 360-degree rotation by means of the steering engine;
the ball screw axial moving device is used for controlling the tail end hooking and cutting actuator to move along the vertical direction; the ball screw transverse moving device is used for controlling the ball screw axial moving device and the tail end hooking and cutting actuator to transversely move;
the five-degree-of-freedom robot is connected to the robot base, the ZED binocular camera is connected with the input end of the controller, so that visual information of the butter in the turtle shell abdomen on the automatic processing production line captured by the ZED binocular camera is transmitted to the controller through a network, and the controller is connected with the execution module, so that the tail end hooking and cutting executor is instructed to reach the butter position to hook and cut and collect the butter in the turtle shell abdomen.
2. The visual positioning and hooking system of butter in an automatic processing line turtle shell abdomen according to claim 1, wherein the visual information data collected by the ZED binocular camera comprises three-dimensional spatial position data, RGB values and depth values of butter.
3. The vision positioning and hooking system for butter in the abdomen of turtle shell of an automatic processing line according to claim 2, characterized in that said ZED binocular camera is fixedly mounted on said knife finger fixing seat; one end of the knife finger fixing seat is fixed on the steering engine flange, and the other end of the knife finger fixing seat is connected with a knife finger; the knife finger controls the opening and closing size by means of the forward and reverse rotation of the stepping motor.
4. The visual positioning and hooking system for butter in the abdomen of turtle shell of claim 3, wherein said five degree of freedom robot is installed on the automated turtle shell processing line; the worm gear device, the ball screw transverse moving device and the ball screw axial moving device are respectively provided with motors for respective control and independent movement; the controller is used for storing and controlling the motion information data of the five-degree-of-freedom robot.
5. The visual positioning and hooking system for butter in an automatic processing line turtle shell abdomen according to claim 4, wherein the tail end hooking actuator is a four-claw type mechanical arm, flexible bionic multi-blade cutters with expandable diameters are welded on the cutter fingers, the width of each flexible bionic multi-blade cutter is set to be 20-30 mm, the number of the cutter fingers is four, and every two adjacent cutter fingers are separated by 90 degrees and are uniformly arranged on the cutter finger fixing seat.
6. A control method of a visual positioning and hooking system of butter in the abdomen of an automatic processing line turtle shell, characterized by comprising the following control steps of:
calibrating a ZED binocular camera to obtain an internal parameter matrix, a distortion coefficient and a re-projection matrix data of the camera, thereby correcting left and right views and converting coordinates;
step two, performing hand-eye calibration on the five-degree-of-freedom robot and the ZED binocular camera to obtain a camera coordinate system and a rotation matrix and a translation matrix of the robot coordinate system;
step three, collecting a large number of turtle shell abdominal butter RGB images in advance, and constructing a turtle shell abdominal butter RGB image sample library;
step four, manually labeling the turtle shell abdominal butter in the image by using an image labeling tool Labelme, manufacturing a turtle shell abdominal butter RGB image data set, and converting the RGB image data set into a Tf record format file;
fifthly, the turtle shell abdominal butter RGB image data set is processed according to the following steps of 6:2:2 is divided into a training set, a verification set and a test set;
training by using a semantic segmentation model deep V3 based on a full convolution neural network to obtain a semantic segmentation model of turtle shell abdominal butter;
fixing the five-degree-of-freedom robot on an automatic processing production line, and adjusting the posture to enable the ZED binocular camera to be opposite to the automatic processing production line;
step eight, opening a ZED binocular camera to acquire an RGB image of the turtle shell belly in real time, calling a pre-trained semantic segmentation model of the turtle shell belly butter to segment the turtle shell belly butter area in real time, and carrying out local stereo matching to calculate the cloud information of the turtle shell belly Huang Youdian;
step nine, calculating the center position coordinates of the turtle shell abdomen butter according to the cloud information of the turtle shell abdomen Huang Youdian;
step ten, controlling the tail end hooking and cutting actuator to move to the position right above the turtle shell abdominal butter according to the calculated center position coordinates of the turtle shell abdominal butter, and then descending and opening the tail end hooking and cutting actuator to finish hooking and cutting actions;
step eleven, after hooking, the tail end hooking actuator ascends and returns to the upper part of the butter collecting box, descends and opens, and the turtle shell abdominal butter is placed into the butter collecting box;
and step twelve, turning the robot back to the upper part of the production line, starting the next round of visual recognition to the butter position in the other turtle shell abdomen, and repeatedly and circularly carrying out the steps until the automatic processing production line stops butter cutting and collecting work.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210031065.7A CN114396889B (en) | 2022-01-12 | 2022-01-12 | Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210031065.7A CN114396889B (en) | 2022-01-12 | 2022-01-12 | Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114396889A CN114396889A (en) | 2022-04-26 |
| CN114396889B true CN114396889B (en) | 2023-07-21 |
Family
ID=81231825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210031065.7A Active CN114396889B (en) | 2022-01-12 | 2022-01-12 | Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114396889B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6681151B1 (en) * | 2000-12-15 | 2004-01-20 | Cognex Technology And Investment Corporation | System and method for servoing robots based upon workpieces with fiducial marks using machine vision |
| CN108834576A (en) * | 2018-06-07 | 2018-11-20 | 华南农业大学 | A kind of citrus picking robot and its implementation based on binocular vision |
| CN109197160A (en) * | 2018-09-12 | 2019-01-15 | 华南农业大学 | A kind of guava picking robot and its implementation |
| CN113196946A (en) * | 2021-06-22 | 2021-08-03 | 华南农业大学 | Self-propelled intelligent fruit and vegetable picking and collecting robot and implementation method thereof |
-
2022
- 2022-01-12 CN CN202210031065.7A patent/CN114396889B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6681151B1 (en) * | 2000-12-15 | 2004-01-20 | Cognex Technology And Investment Corporation | System and method for servoing robots based upon workpieces with fiducial marks using machine vision |
| CN108834576A (en) * | 2018-06-07 | 2018-11-20 | 华南农业大学 | A kind of citrus picking robot and its implementation based on binocular vision |
| CN109197160A (en) * | 2018-09-12 | 2019-01-15 | 华南农业大学 | A kind of guava picking robot and its implementation |
| CN113196946A (en) * | 2021-06-22 | 2021-08-03 | 华南农业大学 | Self-propelled intelligent fruit and vegetable picking and collecting robot and implementation method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114396889A (en) | 2022-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108399639B (en) | Rapid automatic grabbing and placing method based on deep learning | |
| CN112476434A (en) | Visual 3D pick-and-place method and system based on cooperative robot | |
| CN109202912A (en) | A method of objective contour point cloud is registrated based on monocular depth sensor and mechanical arm | |
| CN111515945A (en) | Control method, system and device for mechanical arm visual positioning sorting and grabbing | |
| CN112519978B (en) | Underwater cleaning device and method for ship propeller | |
| CN115553132A (en) | Litchi recognition method based on visual algorithm and bionic litchi picking robot | |
| CN110298885A (en) | Stereoscopic vision identification method and positioning clamping detection device for non-smooth spheroid object and application of stereoscopic vision identification method and positioning clamping detection device | |
| CN112809703A (en) | Bottom sowing sea cucumber catching robot based on ESRGAN enhanced super-resolution and CNN image recognition | |
| CN107363834A (en) | A kind of mechanical arm grasping means based on cognitive map | |
| CN114396889B (en) | Visual positioning and hooking system for butter in turtle shell abdomen of automatic processing line and control method thereof | |
| US11882843B1 (en) | Automated crab meat picking method | |
| Misimi et al. | Robotic handling of compliant food objects by robust learning from demonstration | |
| Vezzani et al. | Improving superquadric modeling and grasping with prior on object shapes | |
| CN111223141B (en) | Automatic pipeline work efficiency optimization system and method based on reinforcement learning | |
| CN109087343A (en) | A kind of generation method and system of workpiece grabbing template | |
| CN114310954A (en) | Self-adaptive lifting control method and system for nursing robot | |
| CN117047237B (en) | Intelligent flexible welding system and method for special-shaped parts | |
| CN114711029B (en) | Vision-positioned large bunch banana fruit combing and picking method and device | |
| Hu et al. | Design and experiment of a new citrus harvesting robot | |
| Ming et al. | Design of porcine abdomen cutting robot system based on binocular vision | |
| CN215011703U (en) | Rose picking robot | |
| CN107199405B (en) | Automatic slicer for corn breeding and sampling | |
| CN114559417A (en) | Kneading machine paddle automatic cleaning robot system based on 3D vision | |
| CN206501118U (en) | A kind of robotic gripping device | |
| CN113119073A (en) | Mechanical arm system based on computer vision and machine learning and oriented to 3C assembly scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |