CN116277025A - Object sorting control method and system of intelligent manufacturing robot - Google Patents
Object sorting control method and system of intelligent manufacturing robot Download PDFInfo
- Publication number
- CN116277025A CN116277025A CN202310420084.3A CN202310420084A CN116277025A CN 116277025 A CN116277025 A CN 116277025A CN 202310420084 A CN202310420084 A CN 202310420084A CN 116277025 A CN116277025 A CN 116277025A
- Authority
- CN
- China
- Prior art keywords
- robot
- intelligent manufacturing
- sorted
- target
- objects
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention provides an object sorting control method and system of an intelligent manufacturing robot, which relate to the field of artificial intelligence and robots and comprise the following steps: acquiring an electronic map model; receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model; based on the information of the objects to be sorted, screening from the object stacking place by adopting an identification technology to obtain target objects, and controlling the intelligent manufacturing robot to perform positioning and grabbing; and controlling the intelligent manufacturing robot to convey the grabbed object to a target storage position according to the optimal path. The optimal path of the robot for conveying the objects to be sorted is obtained through the obtained electronic map model and analysis of the distribution task; based on the to-be-sorted object information obtained by analyzing the distribution task, the robot can identify objects from the object stacking place and perform positioning and grabbing, and finally the to-be-sorted objects are successfully conveyed to the target storage position according to the optimal path.
Description
Technical Field
The invention relates to the technical field of artificial intelligence and robots, in particular to an object sorting control method and system of an intelligent manufacturing robot.
Background
In recent years, with the continuous development of domestic economy, chinese economy is in a mode of taking an optimized economic structure and industrial innovation as a core drive, the external trade is frequent, the market development of electronic commerce is rapid, the logistics industry is prosperous, meanwhile, the goods sorting pressure is greatly increased, the traditional object sorting generally adopts two modes of manual sorting and pipeline platform sorting, the efficiency is lower, the labor cost is higher, and the error rate is also high.
Therefore, the invention provides the object sorting control method and the system for the intelligent manufacturing robot, which improve the object sorting and conveying efficiency and reduce the labor intensity.
Disclosure of Invention
The invention provides an object sorting control method and system of an intelligent manufacturing robot, which are used for obtaining an optimal path for conveying objects to be sorted by the robot through analysis of an obtained electronic map model and an allocation task; based on the to-be-sorted object information obtained by analyzing the distribution task, the robot can identify objects from the object stacking place and perform positioning and grabbing, and finally the to-be-sorted objects are successfully conveyed to the target storage position according to the optimal path.
The invention provides an object sorting control method of an intelligent manufacturing robot, which comprises the following steps:
step 1: acquiring a map of a sorting site and establishing an electronic map model;
step 2: receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
step 3: based on the information of the objects to be sorted, screening from the object stacking place by adopting an identification technology to obtain target objects, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
step 4: and controlling the intelligent manufacturing robot to convey the grabbed object to a target storage position according to the optimal path.
Preferably, receiving and analyzing the distribution task to obtain the information of the objects to be sorted and the information of the target storage position, including:
after receiving the distribution task, triggering a shooting device to acquire an image of an object to be sorted, and meanwhile, analyzing the distribution task to obtain a target storage coordinate position;
removing noise from the image of the object to be sorted by Gaussian filtering;
and extracting key information on the denoised image, and obtaining the information of the objects to be sorted.
Preferably, planning an optimal path of the robot in combination with the electronic map model includes:
based on the electronic map model, gridding the three-dimensional limited motion area of the intelligent manufacturing robot, drawing an obstacle grid and giving a number;
performing global path planning of a grid map with a real-time position of an object to be sorted grasped by a robot as a starting point and a target storage position as an end point based on an ant colony algorithm to obtain a first path;
extracting inflection point data in a first path, sequentially taking inflection points with the distance from the adjacent last inflection point being larger than a preset distance threshold value as segmentation points from a starting point, and dividing the first path into a plurality of first local paths at the segmentation points;
determining the maximum linear speed and the maximum angular speed allowed before the robot brakes when the robot detects that an obstacle exists in front by using an ultrasonic sensor based on the influence factors of the robot for conveying objects;
acquiring the maximum speed allowed by the robot by using the maximum linear speed and the maximum angular speed;
sequentially numbering a starting point, a segmentation point and a terminal point, and sequentially dividing two adjacent points into a group according to the numbers to obtain a plurality of starting and terminal point groups;
taking a robot coordinate system as a reference system, acquiring different first speeds and corresponding first displacements of robots in each starting and ending point group under the omnidirectional movement state based on the constraint of the maximum speed and the preset minimum speed, and projecting the different first speeds and the corresponding first displacements to an electronic map coordinate system to obtain a plurality of available actual displacement tracks, corresponding actual speeds and distances from nearest obstacles;
acquiring different directions of reaching the end points of a plurality of actual displacement tracks acquired by the robot according to the first local path and the start-end point group with consistent start-end point matching respectively, so as to compare and obtain a plurality of angle deviation values;
normalizing the actual speed, the distance of the nearest obstacle and the angle deviation value, calculating to obtain an evaluation value of each actual displacement track, and selecting the actual displacement track with the highest evaluation value as a target displacement track corresponding to the starting and ending point group;
wherein the evaluation value is calculated by the following formula:
wherein P is ij A j-th actual displacement trajectory evaluation value expressed as an i-th start and end point group, wherein j= {1,2,3, …, n }, n being the total number of actual displacement trajectories obtained for each start and end point group; a is that ij The angular deviation value is expressed as the direction when the machine in the ith starting and ending point group reaches the ending point according to the jth actual displacement track and the direction when the machine reaches the ending point according to the first partial path; θ1 is expressed as
The weight coefficient of the angle deviation evaluation function; d (D) ij The distance between the robot and the nearest obstacle on the j-th actual displacement track expressed as the i-th starting and ending point group; θ2 is denoted as->
Weight coefficients of the distance evaluation function; v (V) ij The jth actual displacement trace expressed as the ith starting and ending point groupThe actual speed of the motion of the upper robot; θ3 is expressed as
Weight coefficients of the speed evaluation function;
and splicing all the target displacement tracks obtained by screening to obtain an optimal path, wherein the splicing point is used as an inflection point to be output.
Preferably, based on the information of the objects to be sorted, a target object is obtained by screening from the object stacking place by adopting an identification technology, and the intelligent manufacturing robot is controlled to perform positioning and grabbing, and the method comprises the following steps:
acquiring three-dimensional images of all objects within the visual angle range of the intelligent manufacturing robot by using a binocular camera;
denoising the obtained three-dimensional image by adopting median filtering and three-dimensional correction to obtain a first image;
randomly acquiring a pixel point from a first image as a target point, judging surrounding pixel points in a neighboring area of the target point according to a preset growth rule, merging the surrounding pixel points if the surrounding pixel points meet the preset standard, and stopping growing until the first image is divided into a plurality of sub-images if the surrounding pixel points do not meet the preset standard;
extracting features of the sub-images to obtain first features;
invoking sample features to carry out similarity comparison with the first features to obtain a similarity result;
wherein d i A similarity result expressed as an ith first feature and a sample feature; a, a i1j1 A j1 th class of features denoted as i1 st first features; b j1 Represented as a j 1-th class of features in the sample features; μ is denoted as a feature extraction error factor; n represents the total feature class contained in the i1 th first feature;
analyzing the similarity result, and determining a sub-image corresponding to the maximum similarity as a target image;
and extracting a region to be grabbed of the robot based on the target image, calculating the three-dimensional coordinates of the region to be grabbed, and controlling the intelligent manufacturing robot to carry out positioning grabbing.
Preferably, the extracting the area to be grasped by the robot includes:
step 11: selecting proper threshold parameters according to the light brightness degree of the sorting field, performing binarization processing on the target image, and dividing the top area of the target image;
step 12: selecting a target area through connected area analysis, and carrying out morphological operation treatment on the selected area;
step 13: and extracting the region to be grabbed of the intelligent manufacturing robot based on the minimum circumscribed rectangular region.
Preferably, calculating the three-dimensional coordinates of the region to be grasped includes:
the minimum circumscribed rectangular area is established as a template, and after the template is established successfully, the center coordinates (x z Z) of the matching point is (x) p ,p)；
Based on a preset similarity threshold and constraint conditions, matching judgment is carried out on the center coordinates and the coordinates of the matching points, and if matching is successful, parallax S=is determined z -p；
Obtaining three-dimensional coordinates of the grabbing position by utilizing the acquired parallax;
and based on binocular camera calibration and hand-eye calibration parameters, the three-dimensional coordinates of the grabbing positions are converted into target three-dimensional coordinates of the region to be grabbed under the robot coordinates by using a hand-eye conversion matrix.
Preferably, the method further comprises:
according to the target three-dimensional coordinates, controlling a mechanical arm of the intelligent manufacturing robot to move above the object to be sorted, and judging whether the current object to be sorted can be successfully grabbed by the mechanical arm;
if not, calculating the deflection radian value of the object to be sorted by using the geometric moment of the image;
converting the obtained deflection radian value into an angle, namely a grabbing angle which is required to be adjusted by the mechanical arm clockwise;
and carrying out angle adjustment on the mechanical arm according to the angle, and re-grabbing.
Preferably, controlling the intelligent manufacturing robot to convey the gripped object to the target storage location according to the optimal path includes:
step 21: after the touch sensor is used for determining that the robot successfully grabs the objects to be sorted, a three-dimensional map of an optimal path is called, the data of each inflection point on the optimal path is obtained, and a detection sensor is arranged at each inflection point;
step 22: when the intelligent manufacturing robot enters the vicinity of the inflection point, based on the warning signal sent by the matched detection sensor, the intelligent manufacturing robot is controlled to slow down the current running speed when receiving the warning signal;
step 23: and when the intelligent manufacturing robot cannot receive the warning signal, stably running at a preset speed until the grabbed object is conveyed to the target storage position.
The invention provides an object sorting control system of an intelligent manufacturing robot, which comprises the following components:
the map acquisition module: the method comprises the steps of obtaining a map of a sorting site and establishing an electronic map model;
and a path planning module: the system is used for receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
and a positioning grabbing module: the intelligent manufacturing robot is used for screening the object to be sorted from the object stacking place by adopting an identification technology based on the information of the object to be sorted, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
an object transportation module: and the intelligent manufacturing robot is used for controlling the intelligent manufacturing robot to convey the grabbed objects to the target storage position according to the optimal path.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flow chart of an object sorting control method of an intelligent manufacturing robot according to an embodiment of the invention;
fig. 2 is a block diagram of an object sorting control system of an intelligent manufacturing robot according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
An embodiment of the present invention provides a method for controlling object sorting of an intelligent manufacturing robot, as shown in fig. 1, including:
step 1: acquiring a map of a sorting site and establishing an electronic map model;
step 2: receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
step 3: based on the information of the objects to be sorted, screening from the object stacking place by adopting an identification technology to obtain target objects, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
step 4: and controlling the intelligent manufacturing robot to convey the grabbed object to a target storage position according to the optimal path.
In the embodiment, an electronic map model is established by taking a map of a sorting site as a reference object and is used for subsequent path planning; the content of the distribution task means that the objects to be sorted are successfully transmitted to the appointed storage position; the information of the articles to be sorted comprises color characteristics, size characteristics and shape characteristics; the target storage position information is coordinate position data of specified storage of the objects to be sorted; the optimal path refers to an unobstructed, shortest-time path for the robot to travel to convey the objects to be sorted.
The beneficial effects of the technical scheme are as follows: the optimal path of the robot for conveying the objects to be sorted is obtained through the obtained electronic map model and analysis of the distribution task; based on the to-be-sorted object information obtained by analyzing the distribution task, the robot can identify objects from the object stacking place and perform positioning and grabbing, and finally the to-be-sorted objects are successfully conveyed to the target storage position according to the optimal path, so that the object sorting and conveying efficiency is improved.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which receives and analyzes a distribution task to obtain object information to be sorted and target storage position information, and comprises the following steps:
after receiving the distribution task, triggering a shooting device to acquire an image of an object to be sorted, and meanwhile, analyzing the distribution task to obtain a target storage coordinate position;
removing noise from the image of the object to be sorted by Gaussian filtering;
and extracting key information on the denoised image, and obtaining the information of the objects to be sorted.
In this embodiment, the task of distributing mainly refers to selecting the objects to be sorted from the object stacking place and conveying the objects to the designated storage position; the target storage coordinate position is the position where the object to be sorted should be stored.
In this embodiment, the photographing device mainly refers to a camera; the noise is removed to reduce negative effects of the acquired picture in the later feature extraction caused by the factors of noise of internal circuits of the camera, vibration of mechanical equipment, change of ambient light and the like.
In this embodiment, the key information includes color features, size features, and shape features of the items to be sorted; the item information to be sorted consists of key information, wherein the set of color features, size features and shape features are sample features.
The beneficial effects of the technical scheme are as follows: receiving and analyzing the allocation task to obtain an image of the object to be sorted and a target storage coordinate position; the noise is removed from the image of the object to be sorted to ensure that relatively complete key information can be extracted from the image as much as possible, thereby laying a foundation for subsequent object identification.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which is used for planning an optimal path of the robot by combining an electronic map model, and comprises the following steps:
based on the electronic map model, gridding the three-dimensional limited motion area of the intelligent manufacturing robot, drawing an obstacle grid and giving a number;
performing global path planning of a grid map with a real-time position of an object to be sorted grasped by a robot as a starting point and a target storage position as an end point based on an ant colony algorithm to obtain a first path;
extracting inflection point data in a first path, sequentially taking inflection points with the distance from the adjacent last inflection point being larger than a preset distance threshold value as segmentation points from a starting point, and dividing the first path into a plurality of first local paths at the segmentation points;
determining the maximum linear speed and the maximum angular speed allowed before the robot brakes when the robot detects that an obstacle exists in front by using an ultrasonic sensor based on the influence factors of the robot for conveying objects;
acquiring the maximum speed allowed by the robot by using the maximum linear speed and the maximum angular speed;
sequentially numbering a starting point, a segmentation point and a terminal point, and sequentially dividing two adjacent points into a group according to the numbers to obtain a plurality of starting and terminal point groups;
taking a robot coordinate system as a reference system, acquiring different first speeds and corresponding first displacements of robots in each starting and ending point group under the omnidirectional movement state based on the constraint of the maximum speed and the preset minimum speed, and projecting the different first speeds and the corresponding first displacements to an electronic map coordinate system to obtain a plurality of available actual displacement tracks, corresponding actual speeds and distances from nearest obstacles;
acquiring different directions of reaching the end points of a plurality of actual displacement tracks acquired by the robot according to the first local path and the start-end point group with consistent start-end point matching respectively, so as to compare and obtain a plurality of angle deviation values;
normalizing the actual speed, the distance of the nearest obstacle and the angle deviation value, calculating to obtain an evaluation value of each actual displacement track, and selecting the actual displacement track with the highest evaluation value as a target displacement track corresponding to the starting and ending point group;
wherein the evaluation value is calculated by the following formula:
wherein P is ij A j-th actual displacement trajectory evaluation value expressed as an i-th start and end point group, wherein j= {1,2,3, …, n }, n being the total number of actual displacement trajectories obtained for each start and end point group; a is that ij The angular deviation value is expressed as the direction when the machine in the ith starting and ending point group reaches the ending point according to the jth actual displacement track and the direction when the machine reaches the ending point according to the first partial path; θ1 is expressed as
The weight coefficient of the angle deviation evaluation function; d (D) ij The distance between the robot and the nearest obstacle on the j-th actual displacement track expressed as the i-th starting and ending point group; θ2 is denoted as->
Weight coefficients of the distance evaluation function; v (V) ij The actual speed of the robot motion on the j-th actual displacement track expressed as the i-th starting and ending point group; θ3 is expressed as
Weight coefficients of the speed evaluation function;
and splicing all the target displacement tracks obtained by screening to obtain an optimal path, wherein the splicing point is used as an inflection point to be output.
In the embodiment, the gridding processing has the advantages of simple structure, easy understanding, visual data storage, adjustable model precision, high gridding dividing precision and convenience in path planning; the preset threshold is set in advance.
In this embodiment, the three-dimensional limited motion area refers to a limited area in which the robot can freely move in the plane; the numbering refers to sequentially numbering each grid, which is beneficial to visual understanding and path planning.
In this embodiment, the first path refers to a displacement track obtained after global path planning; the inflection point data comprises inflection point position coordinates, the total number of inflection points and adjacent inflection point intervals; the preset distance threshold is set in advance.
In this embodiment, for example, there are inflection points a1, a2, a3, a4, where a1 and a2 are adjacent, and a3 and a4 are adjacent, where it is determined that the distance between a1 and a2 is greater than the preset distance threshold, and the distance between a3 and a4 is less than the preset distance threshold, so a2 is used as the segmentation point, and the first path is segmented at the inflection point a 2.
In this embodiment, the first local path is a partial displacement track obtained by dividing the first path according to the segment points; factors affected by the robot in conveying objects include the mechanical structure, ground friction and motor performance of the robot; the maximum speed is obtained based on the maximum linear speed and the maximum angular speed allowed before the robot brakes and is used for speed constraint.
In this embodiment, the robot coordinate system is a coordinate system having the robot itself as the origin of coordinates; the preset minimum speed is the minimum running speed of the robot with a preset point in advance; the omnidirectional movement refers to the autorotation action when the robot can move in any direction on a plane; the first speed is the speed value of the robot moving based on the starting point and the ending point in any one starting and ending point group; the first displacement is a track obtained by the robot running at a corresponding first speed based on a starting point and an ending point in any one starting and ending point group; the electronic map coordinate system is a coordinate system established based on a three-dimensional environment area where the robot is located; the actual displacement track is a track obtained by projecting the first displacement on the coordinates of the electronic map; the actual speed is the value of the speed of the robot moving on the corresponding actual displacement track; the distance to the nearest obstacle is the distance of the robot to the nearest obstacle on the corresponding actual displacement trajectory.
In this embodiment, for example, there is a start-end point group 1 including 3 actual displacement tracks, and the corresponding evaluation value P is obtained by calculation 13 >12>11, so the third actual displacement trajectory is taken as the target displacement trajectory of the start and end point group 1.
In this embodiment, the orientation refers to the direction in which the robot is facing when reaching the end point; the angle deviation value refers to the angle difference between the direction opposite to the robot when the robot reaches the end point according to the first local path and the direction opposite to the robot when the robot reaches the end point according to the actual displacement track; the normalization processing aims to avoid the phenomenon that a certain numerical value has excessive influence on the whole evaluation system when the original numerical value is operated.
The beneficial effects of the technical scheme are as follows: obtaining a first path through ant colony algorithm-based planning; the obstacle avoidance capacity is enhanced by carrying out local path planning based on the first path, so that the path smoothing speed and the optimal path searching speed are improved, and the working efficiency of the robot for conveying the objects to be sorted is greatly improved.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which adopts an identification technology to screen a target object from an object stacking place based on object information to be sorted and controls the intelligent manufacturing robot to carry out positioning and grabbing, and comprises the following steps:
acquiring three-dimensional images of all objects within the visual angle range of the intelligent manufacturing robot by using a binocular camera;
denoising the obtained three-dimensional image by adopting median filtering and three-dimensional correction to obtain a first image;
randomly acquiring a pixel point from a first image as a target point, judging surrounding pixel points in a neighboring area of the target point according to a preset growth rule, merging the surrounding pixel points if the surrounding pixel points meet the preset standard, and stopping growing until the first image is divided into a plurality of sub-images if the surrounding pixel points do not meet the preset standard;
extracting features of the sub-images to obtain first features;
invoking sample features to carry out similarity comparison with the first features to obtain a similarity result;
wherein d i A similarity result expressed as an ith first feature and a sample feature; a, a i1j1 A j1 th class of features denoted as i1 st first features; b j1 Represented as a j 1-th class of features in the sample features; μ is denoted as a feature extraction error factor; n represents the total feature class contained in the i1 th first feature;
analyzing the similarity result, and determining a sub-image corresponding to the maximum similarity as a target image;
and extracting a region to be grabbed of the robot based on the target image, calculating the three-dimensional coordinates of the region to be grabbed, and controlling the intelligent manufacturing robot to carry out positioning grabbing.
In the embodiment, median filtering denoising is mainly used for filtering salt and pepper noise; the purpose of stereo correction is to make the left and right image planes coplanar, and the corresponding points on the images are on the same horizontal plane.
In this embodiment, the preset growth rule adopts a region-based gray level difference, i.e., a set threshold, typically 4; the preset standard is that the gray level difference smaller than the set threshold value is judged to be the pixel conforming to the growth rule, and the pixels are combined.
In this embodiment, for example, a pixel a with a gray value of 40 is used as a target point for region growing, and pixels 1,2, and 3 are present in adjacent neighborhoods, and the difference between the gray values is 2,3, and 5, and based on a set threshold value, it is determined that the pixel 1 and the pixel 2 can be combined with the pixel a.
In this embodiment, the sample characteristics refer to color characteristics, size characteristics, and shape characteristics of the items to be sorted.
In this embodiment, for example, the similarity between the presence of the first features b1, b2 and b3 and the features of the sample is d 1 、d 2 、d 3 The method comprises the steps of carrying out a first treatment on the surface of the Wherein d is 2 >d 1 >d 3 The method comprises the steps of carrying out a first treatment on the surface of the The sub-image corresponding to the first feature b2 is the target image.
The beneficial effects of the technical scheme are as follows: acquiring three-dimensional images of all objects in a visual angle range of a robot by using a binocular camera, and carrying out denoising, three-dimensional correction and segmentation processing based on region growth to segment the objects on the three-dimensional images to obtain sub-images; extracting sub-image features to obtain first features; and comparing the similarity of the first features and the sample features to obtain a target image, which is favorable for the extraction of the subsequent region to be grabbed and provides a basis for the positioning and grabbing of the robot.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which is used for extracting a region to be grasped of the robot and comprises the following steps:
step 11: selecting proper threshold parameters according to the light brightness degree of the sorting field, performing binarization processing on the target image, and dividing the top area of the target image;
step 12: selecting a target area through connected area analysis, and carrying out morphological operation treatment on the selected area;
step 13: and extracting the region to be grabbed of the intelligent manufacturing robot based on the minimum circumscribed rectangular region.
In this embodiment, the threshold parameter is selected based on the light intensity of the sorting venue; binarization refers to displaying a target image with an obvious black-and-white effect, and can reflect the whole and local characteristics of the image; the top region segmentation is to facilitate the division of the region to be grasped.
In this embodiment, the connected domain refers to a region composed of pixels having the same pixel value and adjacent in position in the target image after binarization processing; the connected domain analysis refers to finding out mutually independent connected domains in the target image after binarization processing and marking the mutually independent connected domains, wherein the mutually independent connected domains are target areas; morphological operations include swelling and corrosion of structural elements.
In this embodiment, the minimum circumscribed rectangular area refers to the smallest rectangular area in the obtained target area subjected to morphological operation processing, and the object part area corresponding to the smallest rectangular area is the area to be grasped by the intelligent manufacturing robot.
The beneficial effects of the technical scheme are as follows: the region to be grabbed of the robot for intelligent manufacturing is extracted through binarization of the target image, segmentation of the top region, analysis of the connected region and morphological operation processing, accurate grabbing of the robot is facilitated, and efficiency is improved.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which calculates three-dimensional coordinates of a region to be grabbed and comprises the following steps:
the minimum circumscribed rectangular area is established as a template, and after the template is established successfully, the center coordinates (x z ,y z ) The corresponding matching point coordinates are (x p ,y p )；
Based on a preset similarity threshold and constraint conditions, matching judgment is carried out on the center coordinates and the coordinates of the matching points, and if matching is successful, the parallax S=y is determined z -y p ；
Obtaining three-dimensional coordinates of the grabbing position by utilizing the acquired parallax;
and based on binocular camera calibration and hand-eye calibration parameters, the three-dimensional coordinates of the grabbing positions are converted into target three-dimensional coordinates of the region to be grabbed under the robot coordinates by using a hand-eye conversion matrix.
In this embodiment, the minimum circumscribed rectangular area is set up as a template to reduce the influence of illumination variation on the calculation of coordinates of the area to be grasped.
In this embodiment, the preset similarity threshold refers to a similarity value set in advance; in x z ，x p And when the coordinate values are equal, judging that the matching is successful.
In this embodiment, the parallax is acquired to obtain depth information of the object, and a foundation is laid for acquiring three-dimensional coordinates of the region to be grabbed.
In this embodiment, the binocular camera calibration is actually used to obtain parameters of the camera imaging geometric model, that is, a correspondence between the three-dimensional space emphasis and the midpoint of the two-dimensional image; the hand-eye calibration parameters are mainly used for combining the hand-eye conversion matrix, and coordinates of the target object under a robot coordinate system are obtained in the process of grabbing the object by the mechanical arm.
The beneficial effects of the technical scheme are as follows: establishing a template by the extracted region in the target image, returning to the corresponding center coordinate, and acquiring parallax by combining the successfully matched matching point coordinates to obtain the three-dimensional coordinate of the region to be grabbed; the three-dimensional coordinates of the area to be grabbed are converted into the coordinates under the robot coordinate system by utilizing camera calibration and hand-eye calibration, so that the robot can accurately and effectively grab the objects to be sorted.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which further comprises the following steps:
according to the target three-dimensional coordinates, controlling a mechanical arm of the intelligent manufacturing robot to move above the object to be sorted, and judging whether the current object to be sorted can be successfully grabbed by the mechanical arm;
if not, calculating a deflection radian value of the object to be grabbed by using the geometric moment of the image;
converting the obtained deflection radian value into an angle, namely a grabbing angle which is required to be adjusted by the mechanical arm clockwise;
and carrying out angle adjustment on the mechanical arm according to the angle, and re-grabbing.
The beneficial effects of the technical scheme are as follows: the conversion angle of the object to be sorted is calculated by utilizing the geometric moment of the image to determine the angle which the mechanical arm should adjust when moving to the position above the object to be sorted according to the three-dimensional coordinate of the target, so that the robot can be ensured to accurately and effectively grasp the object to be sorted.
The embodiment of the invention provides an object sorting control method of an intelligent manufacturing robot, which is used for controlling the intelligent manufacturing robot to convey a grabbed object to a target storage position according to an optimal path and comprises the following steps:
step 21: after the touch sensor is used for determining that the robot successfully grabs the objects to be sorted, a three-dimensional map of an optimal path is called, the data of each inflection point on the optimal path is obtained, and a detection sensor is arranged at each inflection point;
step 22: when the intelligent manufacturing robot enters the vicinity of the inflection point, based on the warning signal sent by the matched detection sensor, the intelligent manufacturing robot is controlled to slow down the current running speed when receiving the warning signal;
step 23: and when the intelligent manufacturing robot cannot receive the warning signal, stably running at a preset speed until the grabbed object is conveyed to the target storage position.
In the embodiment, the detection sensor is used for sensing and detecting the running speed of the robot, and outputting the warning signal in a form which can be received by the robot, so that the actual speed of the robot can be regulated; the inflection point approach range generally refers to a period of time t before the robot reaches the inflection point and a period of time t leaving the inflection point, wherein the period of time t is determined based on the size of the sorting field and the optimal path distance; the purpose of slowing down current travel speed is in order to guarantee that the robot keeps safe and stable traveling through inflection point department, stops to wait to sort the possibility that the article drops, effectively guarantees that the robot successfully conveys waiting to sort the article.
The beneficial effects of the technical scheme are as follows: after the robot is determined to successfully grasp the objects to be sorted, acquiring an optimal path and data of each inflection point on the optimal path, and setting a detection sensor; the robot carries the object to be sorted and stably runs along the optimal path at a preset speed, when the robot receives the warning signal sent by the detection sensor, the robot is determined to enter the vicinity range of the corresponding inflection point, the current running speed is immediately controlled to be slowed down, the accuracy of determining the speed range of the inflection point region is improved, the actual speed is sensed through the detection sensor, the intelligent speed adjustment is conveniently carried out according to the speed range, and the running stability of the robot in the inflection point region is ensured.
An embodiment of the present invention provides an object sorting control system of an intelligent manufacturing robot, as shown in fig. 2, including:
the map acquisition module: the method comprises the steps of obtaining a map of a sorting site and establishing an electronic map model;
and a path planning module: the system is used for receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
and a positioning grabbing module: the intelligent manufacturing robot is used for screening the object to be sorted from the object stacking place by adopting an identification technology based on the information of the object to be sorted, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
an object transportation module: and the intelligent manufacturing robot is used for controlling the intelligent manufacturing robot to convey the grabbed objects to the target storage position according to the optimal path.
The beneficial effects of the technical scheme are as follows: the optimal path of the robot for conveying the objects to be sorted is obtained through the obtained electronic map model and analysis of the distribution task; based on the to-be-sorted object information obtained by analyzing the distribution task, the robot can identify objects from the object stacking place and perform positioning and grabbing, and finally the to-be-sorted objects are successfully conveyed to the target storage position according to the optimal path, so that the object sorting and conveying efficiency is improved.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. An object sorting control method of an intelligent manufacturing robot is characterized by comprising the following steps:
step 1: acquiring a map of a sorting site and establishing an electronic map model;
step 2: receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
step 3: based on the information of the objects to be sorted, screening from the object stacking place by adopting an identification technology to obtain target objects, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
step 4: and controlling the intelligent manufacturing robot to convey the grabbed object to a target storage position according to the optimal path.
2. The method for controlling object sorting of an intelligent manufacturing robot according to claim 1, wherein receiving and analyzing the distribution task to obtain the object information to be sorted and the target storage location information, comprises:
after receiving the distribution task, triggering a shooting device to acquire an image of an object to be sorted, and meanwhile, analyzing the distribution task to obtain a target storage coordinate position;
removing noise from the image of the object to be sorted by Gaussian filtering;
and extracting key information on the denoised image, and obtaining the information of the objects to be sorted.
3. The method for controlling object sorting of an intelligent manufacturing robot according to claim 2, wherein planning an optimal path of the robot in combination with the electronic map model comprises:
based on the electronic map model, gridding the three-dimensional limited motion area of the intelligent manufacturing robot, drawing an obstacle grid and giving a number;
performing global path planning of a grid map with a real-time position of an object to be sorted grasped by a robot as a starting point and a target storage position as an end point based on an ant colony algorithm to obtain a first path;
extracting inflection point data in a first path, sequentially taking inflection points with the distance from the adjacent last inflection point being larger than a preset distance threshold value as segmentation points from a starting point, and dividing the first path into a plurality of first local paths at the segmentation points;
determining the maximum linear speed and the maximum angular speed allowed before the robot brakes when the robot detects that an obstacle exists in front by using an ultrasonic sensor based on the influence factors of the robot for conveying objects;
acquiring the maximum speed allowed by the robot by using the maximum linear speed and the maximum angular speed;
sequentially numbering a starting point, a segmentation point and a terminal point, and sequentially dividing two adjacent points into a group according to the numbers to obtain a plurality of starting and terminal point groups;
taking a robot coordinate system as a reference system, acquiring different first speeds and corresponding first displacements of robots in each starting and ending point group under the omnidirectional movement state based on the constraint of the maximum speed and the preset minimum speed, and projecting the different first speeds and the corresponding first displacements to an electronic map coordinate system to obtain a plurality of available actual displacement tracks, corresponding actual speeds and distances from nearest obstacles;
acquiring different directions of reaching the end points of a plurality of actual displacement tracks acquired by the robot according to the first local path and the start-end point group with consistent start-end point matching respectively, so as to compare and obtain a plurality of angle deviation values;
normalizing the actual speed, the distance of the nearest obstacle and the angle deviation value, calculating to obtain an evaluation value of each actual displacement track, and selecting the actual displacement track with the highest evaluation value as a target displacement track corresponding to the starting and ending point group;
wherein the evaluation value is calculated by the following formula:
wherein P is ij A j-th actual displacement trajectory evaluation value expressed as an i-th start and end point group, wherein j= {1,2,3, …, n }, n being the total number of actual displacement trajectories obtained for each start and end point group; a is that ij The angular deviation value is expressed as the direction when the machine in the ith starting and ending point group reaches the ending point according to the jth actual displacement track and the direction when the machine reaches the ending point according to the first partial path; θ1 is expressed as
The weight coefficient of the angle deviation evaluation function; d (D) ij The jth actual displacement track represented as the ith starting and ending point group is on machineDistance of the robot from the nearest obstacle; θ2 is denoted as->
Weight coefficients of the distance evaluation function; v (V) ij The actual speed of the robot motion on the j-th actual displacement track expressed as the i-th starting and ending point group; θ3 is expressed as
Weight coefficients of the speed evaluation function;
and splicing all the target displacement tracks obtained by screening to obtain an optimal path, wherein the splicing point is used as an inflection point to be output.
4. The method for controlling object sorting of an intelligent manufacturing robot according to claim 1, wherein the object sorting control method comprises the steps of selecting a target object from an object stacking place by using an identification technology based on object information to be sorted, and controlling the intelligent manufacturing robot to perform positioning and grabbing, and comprises the following steps:
acquiring three-dimensional images of all objects within the visual angle range of the intelligent manufacturing robot by using a binocular camera;
denoising the obtained three-dimensional image by adopting median filtering and three-dimensional correction to obtain a first image;
randomly acquiring a pixel point from a first image as a target point, judging surrounding pixel points in a neighboring area of the target point according to a preset growth rule, merging the surrounding pixel points if the surrounding pixel points meet the preset standard, and stopping growing until the first image is divided into a plurality of sub-images if the surrounding pixel points do not meet the preset standard;
extracting features of the sub-images to obtain first features;
invoking sample features to carry out similarity comparison with the first features to obtain a similarity result;
wherein d i A similarity result expressed as an ith first feature and a sample feature; a, a i1j1 A j1 th class of features denoted as i1 st first features; b j1 Represented as a j 1-th class of features in the sample features; μ is denoted as a feature extraction error factor; n represents the total feature class contained in the i1 th first feature;
analyzing the similarity result, and determining a sub-image corresponding to the maximum similarity as a target image;
and extracting a region to be grabbed of the robot based on the target image, calculating the three-dimensional coordinates of the region to be grabbed, and controlling the intelligent manufacturing robot to carry out positioning grabbing.
5. The method for controlling object sorting of intelligent manufacturing robot according to claim 4, wherein extracting the area to be gripped by the robot comprises:
step 11: selecting proper threshold parameters according to the light brightness degree of the sorting field, performing binarization processing on the target image, and dividing the top area of the target image;
step 12: selecting a target area through connected area analysis, and carrying out morphological operation treatment on the selected area;
step 13: and extracting the region to be grabbed of the intelligent manufacturing robot based on the minimum circumscribed rectangular region.
6. The method for controlling object sorting of intelligent manufacturing robot according to claim 4, wherein calculating three-dimensional coordinates of the area to be grasped comprises:
the minimum circumscribed rectangular area is established as a template, and after the template is established successfully, the center coordinates (x z Z) of the matching point is (x) p ,p)；
Based on a preset similarity threshold and constraint conditions, matching judgment is carried out on the center coordinates and the coordinates of the matching points, and if matching is successful, parallax S=is determined z -p；
Obtaining three-dimensional coordinates of the grabbing position by utilizing the acquired parallax;
and based on binocular camera calibration and hand-eye calibration parameters, the three-dimensional coordinates of the grabbing positions are converted into target three-dimensional coordinates of the region to be grabbed under the robot coordinates by using a hand-eye conversion matrix.
7. The method for controlling object sorting of an intelligent manufacturing robot according to claim 4, further comprising:
according to the target three-dimensional coordinates, controlling a mechanical arm of the intelligent manufacturing robot to move above the object to be sorted, and judging whether the current object to be sorted can be successfully grabbed by the mechanical arm;
if not, calculating the deflection radian value of the object to be sorted by using the geometric moment of the image;
converting the obtained deflection radian value into an angle, namely a grabbing angle which is required to be adjusted by the mechanical arm clockwise;
and carrying out angle adjustment on the mechanical arm according to the angle, and re-grabbing.
8. The method of claim 1, wherein controlling the intelligent manufacturing robot to deliver the gripped objects to the target storage location according to the optimal path comprises:
step 21: after the touch sensor is used for determining that the robot successfully grabs the objects to be sorted, a three-dimensional map of an optimal path is called, the data of each inflection point on the optimal path is obtained, and a detection sensor is arranged at each inflection point;
step 22: when the intelligent manufacturing robot enters the vicinity of the inflection point, based on the warning signal sent by the matched detection sensor, the intelligent manufacturing robot is controlled to slow down the current running speed when receiving the warning signal;
step 23: and when the intelligent manufacturing robot cannot receive the warning signal, stably running at a preset speed until the grabbed object is conveyed to the target storage position.
9. An intelligent manufacturing robot article sorting control system, comprising:
the map acquisition module: the method comprises the steps of obtaining a map of a sorting site and establishing an electronic map model;
and a path planning module: the system is used for receiving and analyzing the distribution task to obtain information of the objects to be sorted and information of target storage positions, and planning an optimal path of the robot by combining the electronic map model;
and a positioning grabbing module: the intelligent manufacturing robot is used for screening the object to be sorted from the object stacking place by adopting an identification technology based on the information of the object to be sorted, and controlling the intelligent manufacturing robot to perform positioning and grabbing;
an object transportation module: and the intelligent manufacturing robot is used for controlling the intelligent manufacturing robot to convey the grabbed objects to the target storage position according to the optimal path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310420084.3A CN116277025A (en) | 2023-04-13 | 2023-04-13 | Object sorting control method and system of intelligent manufacturing robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310420084.3A CN116277025A (en) | 2023-04-13 | 2023-04-13 | Object sorting control method and system of intelligent manufacturing robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116277025A true CN116277025A (en) | 2023-06-23 |
Family
ID=86796091
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310420084.3A Pending CN116277025A (en) | 2023-04-13 | 2023-04-13 | Object sorting control method and system of intelligent manufacturing robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116277025A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935252A (en) * | 2023-07-10 | 2023-10-24 | 齐鲁工业大学(山东省科学院) | Mechanical arm collision detection method based on sub-graph embedded graph neural network |
| CN117160877A (en) * | 2023-11-02 | 2023-12-05 | 启东亦大通自动化设备有限公司 | Article sorting method for logistics robot |
| CN117383126A (en) * | 2023-11-29 | 2024-01-12 | 广州赛志系统科技有限公司 | Plate sorting buffer storage position scheduling method, control system and intelligent sorting production line |
-
2023
- 2023-04-13 CN CN202310420084.3A patent/CN116277025A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116935252A (en) * | 2023-07-10 | 2023-10-24 | 齐鲁工业大学(山东省科学院) | Mechanical arm collision detection method based on sub-graph embedded graph neural network |
| CN116935252B (en) * | 2023-07-10 | 2024-02-02 | 齐鲁工业大学(山东省科学院) | Mechanical arm collision detection method based on sub-graph embedded graph neural network |
| CN117160877A (en) * | 2023-11-02 | 2023-12-05 | 启东亦大通自动化设备有限公司 | Article sorting method for logistics robot |
| CN117160877B (en) * | 2023-11-02 | 2024-01-02 | 启东亦大通自动化设备有限公司 | Article sorting method for logistics robot |
| CN117383126A (en) * | 2023-11-29 | 2024-01-12 | 广州赛志系统科技有限公司 | Plate sorting buffer storage position scheduling method, control system and intelligent sorting production line |
| CN117383126B (en) * | 2023-11-29 | 2024-04-09 | 广州赛志系统科技有限公司 | Plate sorting buffer storage position scheduling method, control system and intelligent sorting production line |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108171748B (en) | Visual identification and positioning method for intelligent robot grabbing application | |
| CN109685066B (en) | Mine target detection and identification method based on deep convolutional neural network | |
| CN116277025A (en) | Object sorting control method and system of intelligent manufacturing robot | |
| CN110344621A (en) | A kind of wheel points cloud detection method of optic towards intelligent garage | |
| CN109460709A (en) | The method of RTG dysopia analyte detection based on the fusion of RGB and D information | |
| CN107392929B (en) | Intelligent target detection and size measurement method based on human eye vision model | |
| KR20180125010A (en) | Control method of mobile robot and mobile robot | |
| CN110334678A (en) | A kind of pedestrian detection method of view-based access control model fusion | |
| CN103959307A (en) | Method of detecting and describing features from an intensity image | |
| CN115049700A (en) | Target detection method and device | |
| CN109001757A (en) | A kind of parking space intelligent detection method based on 2D laser radar | |
| CN111198496A (en) | Target following robot and following method | |
| CN114004814A (en) | Coal gangue identification method and system based on deep learning and gray scale third moment analysis | |
| CN116486287A (en) | Target detection method and system based on environment self-adaptive robot vision system | |
| Jiang et al. | Vision-guided active tactile perception for crack detection and reconstruction | |
| CN102564424A (en) | Multiple sensor-based data fusion method | |
| CN113034575A (en) | Model construction method, pose estimation method and object picking device | |
| CN116309882A (en) | Tray detection and positioning method and system for unmanned forklift application | |
| CN115393696A (en) | Object bin picking with rotation compensation | |
| Budzan | Fusion of visual and range images for object extraction | |
| CN108152824A (en) | A kind of tennis alignment system and method based on laser radar | |
| KR20230061612A (en) | Object picking automation system using machine learning and method for controlling the same | |
| Borgmann et al. | Pedestrian detection and tracking in sparse MLS point clouds using a neural network and voting-based approach | |
| CN114549420A (en) | Workpiece identification and positioning method based on template matching | |
| CN112419337A (en) | Detection method for robot grabbing position under complex background |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |