CN109559341A - A kind of generation method and device of mechanical arm fetching - Google Patents

A kind of generation method and device of mechanical arm fetching Download PDF

Info

Publication number
CN109559341A
CN109559341A CN201710891044.1A CN201710891044A CN109559341A CN 109559341 A CN109559341 A CN 109559341A CN 201710891044 A CN201710891044 A CN 201710891044A CN 109559341 A CN109559341 A CN 109559341A
Authority
CN
China
Prior art keywords
fetching
crawl
point
mechanical arm
crawl point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710891044.1A
Other languages
Chinese (zh)
Other versions
CN109559341B (en
Inventor
王旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Orion Star Technology Co Ltd
Original Assignee
Beijing Orion Star Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Orion Star Technology Co Ltd filed Critical Beijing Orion Star Technology Co Ltd
Priority to CN201710891044.1A priority Critical patent/CN109559341B/en
Publication of CN109559341A publication Critical patent/CN109559341A/en
Application granted granted Critical
Publication of CN109559341B publication Critical patent/CN109559341B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1669—Programme controls characterised by programming, planning systems for manipulators characterised by special application, e.g. multi-arm co-operation, assembly, grasping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/06—Recognition of objects for industrial automation

Abstract

The embodiment of the invention provides the generation methods and device of a kind of mechanical arm fetching, which comprises the point cloud data for obtaining object to be grabbed calculates the centroid position of object to be grabbed；Crawl point range is determined according to preset rules according to centroid position；According to default crawl point-generating algorithm, the corresponding fetching of crawl point in crawl point range is calculated；The corresponding position coordinates of each crawl point are extracted, and determine the crawl point on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance, determine alternative fetching；The corresponding deflection angle of each crawl point is extracted, the corresponding axial drift angle of each alternative fetching is calculated；Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching.The crawl to any attitude, any position object may be implemented, grab efficiency and flexibility is largely increased.

Description

A kind of generation method and device of mechanical arm fetching
Technical field
The present invention relates to mechanical arm data processing fields, a kind of generation method more particularly to mechanical arm fetching and Device.
Background technique
Application scenarios using mechanical arm crawl object are many, for example, automatic sorting cargo, automatic assembling components And robot takes article etc..The crawl that object is carried out using mechanical arm, can save manpower and time, and improve production effect Rate.
Before mechanical arm grabs object, it is necessary first to determine that fetching, fetching include object to be grabbed Position coordinates and mechanical arm tail end be moved to from current location wait grab object space needed for rotation angle, Ke Yili Solution, position coordinates and rotation angle are the data of three-dimensional space, can be indicated with the matrix of a 4*4, i.e. rotation becomes Change matrix.It has been determined that rotational transformation matrix, mechanical arm can be moved and be rotated according to the matrix, and then has completed object Crawl.Crawl of the mechanical arm to object at present, generally requires and manually object is placed in horizontal table top, and is right against mechanical arm End need to only move, the rotation of angle because in this case, the fetching of mechanical arm is relatively simple along a direction Turn also smaller, therefore success rate is higher, robustness is good.
It is clear that can not achieve any attitude in three-dimensional space, any position during the crawl of above-mentioned mechanical arm The crawl of object needs manually to put object, grabs efficiency and flexibility is not high.
Summary of the invention
The generation method and device for being designed to provide a kind of mechanical arm fetching of the embodiment of the present invention, with realization pair The crawl of any attitude, any position object improves crawl efficiency and flexibility.Specific technical solution is as follows:
In a first aspect, the embodiment of the invention provides a kind of generation methods of mechanical arm fetching, which comprises
Obtain the point cloud data of object to be grabbed；
The centroid position of the object to be grabbed is calculated according to the point cloud data；
The crawl point range including the centroid position is determined according to preset rules according to the centroid position；
According to default crawl point-generating algorithm, the crawl point in the crawl point range is coordinately transformed, and according to Coordinate conversion relation calculates the corresponding fetching of crawl point in the crawl point range；
The corresponding position coordinates of each crawl point are extracted from the fetching, and according to the position coordinates, are determined Crawl point on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance, by identified crawl point Corresponding fetching alternately fetching；
Extract the corresponding deflection angle of each crawl point from the alternative fetching, and according to the deflection angle, described Centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching；
Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching.
Optionally, the step of point cloud data for obtaining object to be grabbed, comprising:
Obtain the color image and depth image of the object to be grabbed；
The color image is inputted into preset convolutional neural networks and carries out target detection, the object to be grabbed is obtained and exists Target position in the color image；
Point cloud data is converted by the pixel in the depth image with the target position corresponding part.
Optionally, described that the crawl point including the centroid position is determined according to preset rules according to the centroid position The step of range, comprising:
In three-dimensional space, determine that the range at a distance from the centroid position in the second pre-determined distance is crawl point model It encloses.
Optionally, the default crawl point-generating algorithm of the basis carries out coordinate to the crawl point in the crawl point range Transformation, and according to coordinate conversion relation, the step of calculating the crawl point corresponding fetching in the crawl point range, packet It includes:
According to Generate grasp pose candidates algorithm, to the crawl in the crawl point range The local coordinate system of point carries out rotation transformation；
It is sat according to the rotation transformation relationship of the local coordinate system and the mechanical arm coordinate system and the point cloud data The projection relation of system is marked, the corresponding fetching of each crawl point in the crawl point range is calculated.
Optionally, described that the corresponding deflection angle of each crawl point is extracted from the alternative fetching, and according to described It is inclined to calculate the corresponding axial direction of each alternative fetching for deflection angle, the centroid position and the mechanical arm tail end current location The step of angle, comprising:
The corresponding deflection angle of each crawl point is extracted from the alternative fetching；
According to the centroid position and the mechanical arm tail end current location, calculate the mechanical arm current location relative to The deflection angle of the object to be grabbed；
By the difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point, it is determined as each crawl The corresponding axial drift angle of point.
Optionally, the centroid position is expressed as (x0, y0), the mechanical arm tail end current location is expressed as (x*, y*)；
It is described according to the centroid position and the mechanical arm tail end current location, calculate mechanical arm current location phase For the object to be grabbed deflection angle the step of, comprising:
According to formulaThe mechanical arm current location is calculated relative to the object to be grabbed Deflection angle theta.
Optionally, the method also includes:
Control the crawl that the mechanical arm carries out object to be grabbed according to the target fetching.
Second aspect, the embodiment of the invention provides a kind of generating means of mechanical arm fetching, described device includes:
Point cloud data obtains module, for obtaining the point cloud data of object to be grabbed；
Centroid position determining module, for calculating the centroid position of the object to be grabbed according to the point cloud data；
Point range determining module is grabbed, for according to preset rules, determining to include the mass center according to the centroid position The crawl point range of position；
Fetching determining module, for grabbing point-generating algorithm according to default, to the crawl in the crawl point range Point is coordinately transformed, and according to coordinate conversion relation, calculates the corresponding fetching of crawl point in the crawl point range；
Alternative fetching determining module is sat for extracting the corresponding position of each crawl point from the fetching Mark, and according to the position coordinates, it determines on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance Interior crawl point, by the corresponding fetching of identified crawl point alternately fetching；
Axial drift angle determining module, for extracting the corresponding deflection angle of each crawl point from the alternative fetching, And according to the deflection angle, the centroid position and the mechanical arm tail end current location, each alternative fetching pair is calculated The axial drift angle answered；
Target fetching generation module, for alternative fetching corresponding to the smallest axial drift angle to be determined as mesh Mark fetching.
Optionally, the point cloud data acquisition module includes:
Image acquisition unit, for obtaining the color image and depth image of the object to be grabbed；
Target position determination unit carries out target inspection for the color image to be inputted preset convolutional neural networks It surveys, obtains the target position of the object to be grabbed in the color image；
Point cloud data acquiring unit, for will turn in the depth image with the pixel of the target position corresponding part Turn to point cloud data.
Optionally, the crawl point range determining module includes:
Point range determination unit is grabbed, it is default second at a distance from the centroid position for determining in three-dimensional space Range in distance is crawl point range.
Optionally, the fetching determining module includes:
Rotation transformation unit is used for according to Generate grasp pose candidates algorithm, to described The local coordinate system for grabbing the crawl point in point range carries out rotation transformation；
Fetching determination unit, for according to the local coordinate system rotation transformation relationship and the mechanical arm The projection relation of coordinate system and the point cloud data coordinate system calculates the corresponding crawl of each crawl point in the crawl point range Scheme.
Optionally, the axial drift angle determining module includes:
First deflection angle determination unit, for extracting the corresponding deflection of each crawl point from the alternative fetching Angle；
Second deflection angle determination unit, for calculating according to the centroid position and the mechanical arm tail end current location Deflection angle of the mechanical arm current location relative to the object to be grabbed；
Axial drift angle determination unit is used for the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point Difference, be determined as the corresponding axial drift angle of each crawl point.
Optionally, the centroid position is expressed as (x0, y0), the mechanical arm tail end current location is expressed as (x*, y*)；
The second deflection angle determination unit includes:
Deflection angle determines subelement, for according to formulaCalculate the mechanical arm current location Deflection angle theta relative to the object to be grabbed.
Optionally, described device further include:
Control module is grabbed, carries out grabbing for object to be grabbed according to the target fetching for controlling the mechanical arm It takes.
The third aspect, the embodiment of the invention provides a kind of electronic equipment, including processor, communication interface, memory and Communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes the generation side of above-mentioned mechanical arm fetching Method step.
The third aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage Dielectric memory contains computer program, and above-mentioned mechanical arm fetching is realized when the computer program is executed by processor Generation method step.
In scheme provided by the embodiment of the present invention, the point cloud data of object to be grabbed is obtained first, according to point cloud data The centroid position for calculating object to be grabbed determines the crawl point model including centroid position according to preset rules according to centroid position It encloses, then according to default crawl point-generating algorithm, the crawl point in crawl point range is coordinately transformed, and calculate crawl point The corresponding fetching of crawl point in range, then the corresponding position coordinates of each crawl point are extracted from fetching, and root According to the position coordinates, the crawl on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance is determined Point finally extracts the corresponding fetching of identified crawl point alternately fetching every from alternative fetching The corresponding deflection angle of a crawl point, and according to deflection angle, centroid position and mechanical arm tail end current location, calculating is each alternatively grabbed The corresponding axial drift angle of scheme is taken, alternative fetching corresponding to the smallest axial drift angle is determined as target fetching. Since target fetching is to the crawl point in crawl point range after three-dimensional space is coordinately transformed, by distance screening And axial drift angle screening is determining, therefore the crawl to any attitude, any position object may be implemented, and grabs efficiency and flexibly Property is largely increased.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow chart of the generation method of mechanical arm fetching provided by the embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of point cloud data；
Fig. 3 is the specific flow chart that the point cloud data mode of object to be grabbed is obtained in embodiment illustrated in fig. 1；
Fig. 4 is the specific flow chart that axial drift angle mode is determined in embodiment illustrated in fig. 1；
Fig. 5 is a kind of schematic diagram of mechanical arm relative to the deflection angle of object to be grabbed；
Fig. 6 is a kind of structural schematic diagram of the generating means of mechanical arm fetching provided by the embodiment of the present invention；
Fig. 7 is the structural schematic diagram of a kind of electronic equipment provided by the embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.
In order to realize the crawl to any attitude, any position object, crawl efficiency and flexibility are improved, the present invention is implemented Example provides generation method, device, electronic equipment and the computer readable storage medium of a kind of mechanical arm fetching.
The generation method for being provided for the embodiments of the invention a kind of mechanical arm fetching first below is introduced.
What it is firstly the need of explanation is, it is necessary first to explanation, a kind of mechanical arm crawl provided by the embodiment of the present invention The generation method of scheme can be applied to any electronic equipment (hereinafter referred to as electronic equipment) that communication connection is established with mechanical arm, It is understood that data and instruction can be sent between the electronic equipment and mechanical arm.The electronic equipment can be computer, place The electronic equipments such as device are managed, are not specifically limited herein.Under normal circumstances, camera is installed on the mechanical arm, for being wanted The object (object i.e. to be grabbed) of crawl is shot, and image is acquired.
As shown in Figure 1, a kind of generation method of mechanical arm fetching, which comprises
S101 obtains the point cloud data of object to be grabbed；
It is understood that point cloud data is the data being made of the spatial point for the object being able to detect that in space, it can To characterize the position of object in space.Each spatial point in point cloud data is the coordinate representation for passing through it in space.Cause This point cloud data for obtaining object to be grabbed has determined the position in space of object to be grabbed.
In one embodiment, point cloud data can be obtained by 3D scanning device, for example, laser radar, solid are taken the photograph As first-class equipment.Specifically, crawl object is treated using above-mentioned 3D scanning device to be scanned, can be obtained object to be grabbed Point cloud data.
In another embodiment, since the color image of object can characterize position of the object in plane space, And depth image can characterize object in the position in the plane space direction characterized perpendicular to color image, it therefore, can basis The color image and depth image of object to be grabbed determine the point cloud data of object to be grabbed.It is clear in order to be laid out clear and scheme Chu, it is subsequent to carry out citing introduction to which.
It is of course also possible to use other obtain the mode of point cloud data in the related technology, as long as available object to be grabbed The point cloud data of body, is not specifically limited herein.Point cloud data shown in Fig. 2 is the signal of the point cloud data of a cup Figure, it is seen then that in point cloud data, the information such as position, shape, the profile of object can be shown, convenient for the life of subsequent fetching At.
S102 calculates the centroid position of the object to be grabbed according to the point cloud data；
Since the centroid position of object is the center of mass of object distribution, it is to be understood that for general object For, when being grabbed, crawl position, then can be more stable then grabbing more close to centroid position.So, electronic equipment obtains After the point cloud data for grabbing object, the centroid position of object to be grabbed can be calculated according to point cloud data.
Specifically, the point cloud data of object to be grabbed includes n spatial point, and coordinate is respectively (x1, y1, z1)、(x2, y2, z2)、(x3, y3, z3)…(xn, yn, zn).The centroid position of object so to be grabbed is
For example, if the point cloud data of object to be grabbed include 5 spatial points, coordinate be respectively (5,8,7), (10, 2,4), (5,7,10), (12,6,34), (21,5,9).The centroid position of object so to be grabbed is
It should be noted that the point cloud data of object to be grabbed includes 5 spatial points in above-mentioned example, it is intended merely to clear The calculation of centroid position compactly described, under actual conditions, spatial point quantity that the point cloud data of object to be grabbed includes Be that very more, as shown in Figure 2 schematic diagrames can be found out, but no matter the space that the point cloud data of object to be grabbed includes Point quantity is how many, and it is constant for calculating the mode of centroid position.
S103 determines the crawl point range including the centroid position according to preset rules according to the centroid position；
It is understood that the crawl position of mechanical arm is not necessarily object to be grabbed grabbing when grabbing object Centroid position, as long as be in centroid position a certain range can be successful, and in this way for mechanical arm grab Difficulty also available reduction.In addition, for some objects in irregular shape, such as some machine components, ornament etc., Its centroid position may not be on object, therefore, after electronic equipment has determined above-mentioned centroid position, can be to according to mass center position It sets, determines a reasonable crawl point range.That is, electronic equipment can determine to include the mass center according to preset rules The crawl point range of position.
In one embodiment, electronic equipment can be in three-dimensional space, determination and the centroid position of object to be grabbed Range of the distance in the second pre-determined distance is crawl point range.It is understood that crawl determined by electronic equipment in this way Point range is on three-dimensional space using the centroid position as the centre of sphere, and the second pre-determined distance is a spheric region of the radius of a ball, The range for being likely to become crawl point can be determined in this way, convenient for being subsequently generated fetching.
It should be noted that above-mentioned second pre-determined distance can be determined according to the actual size of object to be grabbed, herein It is not specifically limited.If the small volume of object to be grabbed, the second pre-determined distance can be smaller, for example, 2 centimetres, 3 lis Rice, 5 centimetres etc., it is excessive to avoid crawl point range, cause the fetching of subsequent determination not accurate enough.If object to be grabbed Volume it is larger, then the second pre-determined distance can be larger, for example, 7 centimetres, 8 centimetres, 10 centimetres etc., so that the crawl determined Point range includes all possible crawl point as far as possible.
S104 is coordinately transformed the crawl point in the crawl point range according to default crawl point-generating algorithm, and According to coordinate conversion relation, the corresponding fetching of crawl point in the crawl point range is calculated；
It, can be according to default crawl point-generating algorithm, to the crawl point model after electronic equipment determines above-mentioned crawl point range Crawl point in enclosing is coordinately transformed, and then according to coordinate conversion relation, the crawl point calculated in the crawl point range is corresponded to Fetching.Wherein, Generate grasp pose candidates can be used by presetting crawl point-generating algorithm Algorithm etc. grabs point-generating algorithm.
In order to simulate any attitude of object to be grabbed in three dimensions, electronic equipment can give birth to according to default crawl point At algorithm, the crawl point in the crawl point range is coordinately transformed, such as coordinate system rotation etc., to simulate object to be grabbed Different postures in three dimensions can obtain the information of different postures, different crawls point position in this way.
In turn, electronic equipment can calculate each crawl point pair according to the corresponding coordinate conversion relation of each crawl point The fetching answered.It is subsequent each to calculating to grab the corresponding fetching of point in order to which scheme understands and is laid out clearly Concrete mode carries out citing introduction.
S105, extracts the corresponding position coordinates of each crawl point from the fetching, and according to the position coordinates, It determines the crawl point on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance, is grabbed identified Take a little corresponding fetching alternately fetching；
The corresponding fetching of crawl point since there are randomnesss for above-mentioned coordinate transform, after may result in coordinate transform Stable crawl is difficult to realize in actually crawl, therefore the Stability and veracity in order to guarantee crawl, electronic equipment are obtaining After obtaining above-mentioned fetching, fetching can be filtered, filter out some crawls more difficult for mechanical arm crawl Scheme.
That is, electronic equipment can determine on preset coordinate direction, with mechanical arm tail end current location first Crawl point in pre-determined distance, by the corresponding fetching of identified crawl point alternately fetching.In general, should Preset coordinate direction can be the vertical direction of mechanical arm coordinate system, indicated with z-axis, can guarantee mechanical arm in z-axis in this way The moving distance in direction is not too big.Certainly, which may be other directions, specifically can be according to mechanical arm The factors such as usage scenario and the structure of mechanical arm itself determine, are not specifically limited herein.
Above-mentioned first pre-determined distance can be set according to factors such as the sizes of practical crawl environment and object to be grabbed, such as It can be 2 centimetres, 3 centimetres, 5 centimetres etc., be not specifically limited herein.
S106, extracts the corresponding deflection angle of each crawl point from the alternative fetching, and according to the deflection angle, The centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching；
The processing of S105 through the above steps, available some alternative fetchings, in turn, electronic equipment can be from The middle best fetching of determination.Since the rotation angle of mechanical arm will have a direct impact on the accuracy and stability of crawl, then electricity Sub- equipment can choose best fetching according to the corresponding axial drift angle of crawl point each in alternative fetching.
Electronic equipment can extract the corresponding deflection angle of each crawl point from alternative fetching, then according to deflection Angle, above-mentioned centroid position and mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching.
Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching by S107.
After the corresponding axial drift angle of each alternative fetching has been determined, electronic equipment can therefrom determine the smallest axial direction Drift angle, since axial drift angle is smaller, the angle that mechanical arm grabs required rotation when grabbing object is smaller, and crawl difficulty is lower, Success rate and accuracy are also higher, so, electronic equipment can be by alternative fetching corresponding to the smallest axial drift angle It is determined as target fetching.
It should be noted that the reference of target fetching is to determine that mechanical arm is carried out wait grab by the above method The fetching that can be used when the crawl of object is taken, and does not have other limiting meanings.
As it can be seen that electronic equipment obtains the point cloud number of object to be grabbed first in the scheme provided by the embodiment of the present invention According to, the centroid position of object to be grabbed is calculated according to point cloud data, it is determining including matter according to preset rules according to centroid position The crawl point range of heart position carries out coordinate to the crawl point in crawl point range then according to default crawl point-generating algorithm Transformation, and the corresponding fetching of crawl point in crawl point range is calculated, then each crawl point pair is extracted from fetching Position coordinates answered, and according to the position coordinates determine on preset coordinate direction with mechanical arm tail end current location the Crawl point in one pre-determined distance, by the corresponding fetching of identified crawl point alternately fetching, finally from standby It selects and extracts the corresponding deflection angle of each crawl point in fetching, and is current according to deflection angle, centroid position and mechanical arm tail end Position calculates the corresponding axial drift angle of each alternative fetching, and the corresponding alternative fetching in the smallest axial direction drift angle is true It is set to target fetching.Since target fetching is to carry out coordinate change in three-dimensional space to the crawl point in crawl point range After changing, determining by distance screening and axial drift angle screening, therefore may be implemented to grab any attitude, any position object It takes, grabs efficiency and flexibility is largely increased.
As a kind of embodiment of the embodiment of the present invention, as shown in figure 3, the above-mentioned point cloud data for obtaining object to be grabbed The step of, may include:
S301 obtains the color image and depth image of the object to be grabbed；
Under normal circumstances, electronic equipment can treat crawl object by color camera and depth camera simultaneously and carry out Shooting, and then can be in the color image and depth image for obtaining object to be grabbed.
It is understood that color camera and depth camera are generally mounted in mechanical arm tail end, such depth Image characterizes object to be grabbed at a distance from mechanical arm tail end.
The color image is inputted preset convolutional neural networks and carries out target detection by S302, is obtained described wait grab Target position of the object in the color image；
Electronic equipment obtains after the color image for grabbing object, can be inputted preset convolutional neural networks into Row target detection, and then target position of the object to be grabbed in the color image can be obtained.
Wherein, preset convolutional neural networks can be the convolutional neural networks for being arbitrarily able to carry out target detection.It can be with Understand, which is to be obtained by preset convolutional neural networks model by parameter training, by It can be using parameter training mode in the related technology, such as backpropagation in carrying out parameter training to convolutional neural networks model Method etc., is not specifically limited herein and illustrates.
For example, convolutional neural networks model can be established based on Caffe deep learning frame, then by above-mentioned color The object to be grabbed of camera acquisition is at various locations and the image under various forms is as image pattern, and demarcates each image The position of object to be grabbed in sample, it is general to be indicated using a rectangle frame, then by image pattern and its corresponding wait grab The position of object inputs preset convolutional neural networks model, carries out parameter adjustment, until iteration reach for several times preset times or The convergence of person's objective function, can stop parameter training, obtain preset convolutional neural networks, which is Learn the feature to object to be grabbed.
In turn, when color image is inputted preset convolutional neural networks progress target detection by electronic equipment, preset volume Product neural network can obtain object to be grabbed in the color according to the feature of its object to be grabbed learnt in training Target position and output in image, electronic equipment just obtain target position of the object to be grabbed in the color image.
The pixel in the depth image with the target position corresponding part is converted point cloud data by S303.
Above-mentioned target position is generally a rectangle frame, in this way, after electronic equipment obtains the target position, it can be upper It states and determines corresponding position in depth image.It is understood that color image is usually that size is identical with depth image, it include picture Identical two images of vegetarian refreshments quantity, then in depth image with part that target position corresponding part is in rectangle frame.Example Such as, target position is (xa,ya,xb,yb), wherein (xa,ya) and (xb,yb) it is two, the rectangle coordinates to angular vertex, then depth It is in depth image with target position corresponding part in image, two coordinates to angular vertex are (xa,ya) and (xb,yb) Rectangle.
Next, electronic equipment can will be converted into a cloud with the pixel of target position corresponding part in depth image Data.Position of the coordinate representation pixel of pixel in camera coordinate system in depth image, and the gray value of pixel Indicate the corresponding depth of pixel, as pixel is at a distance from mechanical arm tail end, that is, camera coordinate system corresponding three Another coordinate value in dimension space.So (x, y, z) of the coordinate of pixel and gray value composition is camera coordinates Point cloud data under system.In this way, the point cloud data of object to be grabbed can be obtained.
As it can be seen that the point cloud data for obtaining object to be grabbed through the above way does not need the equipment using profession, need to only make With common color camera and depth camera, cost is reduced.Moreover, easy to operate, calculation amount is small.
As a kind of embodiment of the embodiment of the present invention, the default crawl point-generating algorithm of above-mentioned basis, to the crawl Crawl point in point range is coordinately transformed, and according to coordinate conversion relation, calculates the crawl point in the crawl point range The step of corresponding fetching may include:
According to Generate grasp pose candidates algorithm, to the crawl in the crawl point range The local coordinate system of point carries out rotation transformation；It is sat according to the rotation transformation relationship of the local coordinate system and the mechanical arm Mark system and the projection relation of the point cloud data coordinate system calculate the corresponding crawl side of each crawl point in the crawl point range Case.
Electronic equipment can use Generate grasp pose candidates algorithm, to above-mentioned crawl point The local coordinate system of crawl point in range carries out rotation transformation, specifically, can randomly select from crawl point range first Multiple crawl points calculate the normal and local coordinate system of the crawl point, then by each local coordinate system at each crawl point It is arbitrarily rotated along an axis, simulates various gestures of object to be grabbed.
Then electronic equipment can be according to the rotation transformation relationship and mechanical arm coordinate system and point cloud number of local coordinate system According to the projection relation of coordinate system, the corresponding fetching of each crawl point in crawl point range is calculated.It had been coordinately transformed Cheng Zhong can recorde the projection relation matrix of coordinate transform, be the matrix of a 4*4, such asSo matrix is left The 3*3 element at upper angleThat is indicates coordinate rotation relationship, elementThat is indicates coordinate translation relation.It is understood that should Matrix is position and the angular relationship for characterizing crawl point and being coordinately transformed front and back, if for mechanical arm, machine Tool arm need to only carry out movement i.e. to be moved to the crawl point after coordinate transform from the crawl point before coordinate transform according to the matrix Can, the matrix i.e. mechanical arm are grabbed from crawl point corresponding one put after being moved to coordinate transform that grabs before coordinate transform Take scheme.
Since coordinate system where the point cloud data of object to be grabbed is usually the coordinate system for obtaining point cloud data equipment, example Such as, if point cloud data is obtained by color image and depth image, then point cloud data is under camera coordinate system Data.Therefore, in the corresponding fetching of each crawl point of calculating, it is also necessary to which the crawl point after coordinate transform is corresponding Fetching is converted to mechanical arm coordinate system, and mechanical arm could be moved according to fetching.
Specifically, still by taking point cloud data is obtained by color image and depth image as an example, then video camera Projection relation can be obtained by camera calibration technology between coordinate system and mechanical arm coordinate system, also be the matrix of a 4*4, Such asSo by the projection relation matrix of above-mentioned coordinate transform and the matrix multiple, i.e.,The corresponding fetching of crawl point after coordinate transform can be turned It shifts under mechanical arm coordinate system, has also just obtained the corresponding fetching of crawl point
All crawl points in crawl point range are converted in the manner described above, can be obtained in crawl point range The corresponding fetching of all crawl points, these fetchings are the fetching under mechanical arm coordinate system.
As it can be seen that by any rotation transformation of the local coordinate system to the crawl point randomly selected, it is available wait grab The different postures of object are applicable to various postures in the different information for grabbing point positions, the fetching being calculated in this way With the object to be grabbed of position.
As a kind of embodiment of the embodiment of the present invention, as shown in figure 4, above-mentioned extract from the alternative fetching The corresponding deflection angle of each crawl point, and according to the deflection angle, the centroid position and the mechanical arm tail end current location, The step of calculating each alternative fetching corresponding axial drift angle may include:
S401 extracts the corresponding deflection angle of each crawl point from the alternative fetching；
Due to the matrix that fetching is a 4*4, and 3*3 element representation for as above giving an account of its upper left corner that continues is rotation Transfer the registration of Party membership, etc. from one unit to another, then the corresponding deflection angle of crawl point can be calculated according to the 3*3 element.For example, can be according to Euler The transformational relation of angle and matrix extracts deflection angle, that is, the corresponding deflection angle of each crawl point from alternative fetching.
It should be noted that each crawl point corresponding deflection angle characterization is under mechanical arm coordinate system, mechanical arm is worked as When preceding deflection angle is 0 degree, the angle of required rotation when putting corresponding deflection angle to each crawl is rotated.
S402 calculates the mechanical arm current location according to the centroid position and the mechanical arm tail end current location Deflection angle relative to the object to be grabbed；
In actual use due to mechanical arm, current location is frequently not 0 degree relative to the deflection angle of object to be grabbed, institute After the corresponding deflection angle of each crawl point has been determined, also need to determine deflection of the mechanical arm current location relative to object to be grabbed Angle just can determine that the corresponding axial drift angle of each crawl point.
So, electronic equipment can be according to above-mentioned centroid position and mechanical arm tail end current location, and calculating machine arm is current Deflection angle of the position relative to object to be grabbed.It is understood that the rotation angle of mechanical arm is generally three-dimensional data, in order to It quickly and easily determines axial drift angle, deflection of the mechanical arm on some direction initialization relative to object to be grabbed can be used Angle calculates.In order to keep mechanical arm crawl difficulty minimum, can be used as using the highest direction of the practical rotation difficulty of mechanical arm should Direction initialization.So above-mentioned from the corresponding deflection angle of each crawl point is extracted in alternative fetching is also on the direction initialization Deflection angle.
As in one embodiment, the direction of the plane determined with reference axis x, y indicates above-mentioned direction initialization, then If the centroid position of object to be grabbed is expressed as (x0, y0), mechanical arm tail end current location is expressed as (x*, y*), it can root According to formulaCalculate deflection angle theta of the mechanical arm current location relative to object to be grabbed.
As shown in Figure 5, wherein by the geometrical relationship of mechanical arm 501 and object to be grabbed 502 it is found that deflection angle theta is in machinery Sine value under arm coordinate system isThe value of so deflection angle is
The difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point is determined as every by S403 The corresponding axial drift angle of a crawl point.
It is to be appreciated that the difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point, as mechanical Arm is rotated to the angle of actual needs rotation when each crawl, referred to as axial drift angle.It is determined that each crawl point corresponds to Deflection angle and the corresponding deflection angle of mechanical arm, electronic equipment also can be by the corresponding deflection angle of mechanical arm and each crawl point The difference of corresponding deflection angle is determined as the corresponding axial drift angle of each crawl point.
As a kind of embodiment of the embodiment of the present invention, the above method can also include:
Control the crawl that the mechanical arm carries out object to be grabbed according to the target fetching.
After electronic equipment has determined above-mentioned target fetching, mechanical arm can be controlled according to the target fetching pair Object to be grabbed is grabbed.Specifically, target fetching is the matrix of a 4*4, and electronic equipment can control machinery Arm is rotated according to the element numerical value of the characterization rotation relationship in the matrix of the 4*4, flat according to the characterization in the matrix of the 4*4 The element numerical value of shifting relationship is translated, and then is moved at object to be grabbed, and grasping movement is carried out.
Corresponding to above method embodiment, the embodiment of the invention also provides a kind of generation of mechanical arm fetching dresses It sets.
The generating means for being provided for the embodiments of the invention a kind of mechanical arm fetching below are introduced.
As shown in fig. 6, a kind of generating means of mechanical arm fetching, described device include:
Point cloud data obtains module 610, for obtaining the point cloud data of object to be grabbed；
Centroid position determining module 620, for calculating the centroid position of the object to be grabbed according to the point cloud data；
Point range determining module 630 is grabbed, for according to preset rules, determining to include described according to the centroid position The crawl point range of centroid position；
Fetching determining module 640, for grabbing grabbing in point range to described according to default crawl point-generating algorithm It takes and is a little coordinately transformed, and according to coordinate conversion relation, calculate the corresponding crawl side of crawl point in the crawl point range Case；
Alternative fetching determining module 650, for extracting the corresponding position of each crawl point from the fetching Coordinate, and according to the position coordinates, determine on preset coordinate direction with mechanical arm tail end current location first it is default away from From interior crawl point, by the corresponding fetching of identified crawl point alternately fetching；
Axial drift angle determining module 660, for extracting the corresponding deflection of each crawl point from the alternative fetching Angle, and according to the deflection angle, the centroid position and the mechanical arm tail end current location, calculate each alternative fetching Corresponding axial direction drift angle；
Target fetching generation module 670, for determining alternative fetching corresponding to the smallest axial drift angle For target fetching.
As it can be seen that the point cloud data of object to be grabbed is obtained in the scheme provided by the embodiment of the present invention, according to a cloud number According to the centroid position for calculating object to be grabbed, the crawl point including centroid position is determined according to preset rules according to centroid position Range is coordinately transformed the crawl point in crawl point range, and calculate crawl then according to default crawl point-generating algorithm The corresponding fetching of crawl point in point range, then the corresponding position coordinates of each crawl point are extracted from fetching, and According to the position coordinates, determination grabbing in the first pre-determined distance with mechanical arm tail end current location on preset coordinate direction It takes a little, the corresponding fetching of identified crawl point alternately fetching is finally extracted from alternative fetching The corresponding deflection angle of each crawl point, and according to deflection angle, centroid position and mechanical arm tail end current location, it calculates each alternative The corresponding alternative fetching in the smallest axial direction drift angle is determined as target fetching by the corresponding axial drift angle of fetching. Since target fetching is to the crawl point in crawl point range after three-dimensional space is coordinately transformed, by distance screening And axial drift angle screening is determining, therefore the crawl to any attitude, any position object may be implemented, and grabs efficiency and flexibly Property is largely increased.
As a kind of embodiment of the embodiment of the present invention, the point cloud data obtains module 610 and may include:
Image acquisition unit (is not shown) in Fig. 6, for obtaining the color image and depth image of the object to be grabbed；
Target position determination unit (is not shown) in Fig. 6, for the color image to be inputted preset convolutional Neural net Network carries out target detection, obtains the target position of the object to be grabbed in the color image；
Point cloud data acquiring unit (is not shown) in Fig. 6, and being used for will be corresponding with the target position in the depth image Partial pixel is converted into point cloud data.
As a kind of embodiment of the embodiment of the present invention, the crawl point range determining module 630 may include:
Grab point range determination unit (being not shown in Fig. 6), in three-dimensional space, it is determining with the centroid position away from It is crawl point range from the range in the second pre-determined distance.
As a kind of embodiment of the embodiment of the present invention, the fetching determining module 640 may include:
Rotation transformation unit (is not shown) in Fig. 6, for according to Generate grasp pose candidates Algorithm carries out rotation transformation to the local coordinate system of the crawl point in the crawl point range；
Fetching determination unit (is not shown) in Fig. 6, for the rotation transformation relationship according to the local coordinate system, with And the projection relation of the mechanical arm coordinate system and the point cloud data coordinate system, calculate each crawl in the crawl point range The corresponding fetching of point.
As a kind of embodiment of the embodiment of the present invention, axial direction drift angle determining module 660 may include:
First deflection angle determination unit (being not shown in Fig. 6), for extracting each crawl from the alternative fetching The corresponding deflection angle of point；
Second deflection angle determination unit (being not shown in Fig. 6), for according to the centroid position and the mechanical arm tail end Current location calculates deflection angle of the mechanical arm current location relative to the object to be grabbed；
Axial drift angle determination unit (being not shown in Fig. 6) is used for the corresponding deflection angle of the mechanical arm and each crawl The difference of the corresponding deflection angle of point is determined as the corresponding axial drift angle of each crawl point.
As a kind of embodiment of the embodiment of the present invention, the centroid position can be expressed as (x0, y0), the machinery Arm end current location can be expressed as (x*, y*)；
The second deflection angle determination unit may include:
Deflection angle determines subelement (being not shown in Fig. 6), for according to formulaCalculate the machine Deflection angle theta of the tool arm current location relative to the object to be grabbed.
As a kind of embodiment of the embodiment of the present invention, described device can also include:
It grabs control module (being not shown in Fig. 6), is carried out for controlling the mechanical arm according to the target fetching The crawl of object to be grabbed.
The embodiment of the invention also provides a kind of electronic equipment, as shown in fig. 7, comprises processor 701, communication interface 702, Memory 703 and communication bus 704, wherein processor 701, communication interface 702, memory 703 are complete by communication bus 704 At mutual communication,
Memory 703, for storing computer program；
Processor 701 when for executing the program stored on memory 703, realizes following steps:
Obtain the point cloud data of object to be grabbed；
The centroid position of the object to be grabbed is calculated according to the point cloud data；
The crawl point range including the centroid position is determined according to preset rules according to the centroid position；
According to default crawl point-generating algorithm, the crawl point in the crawl point range is coordinately transformed, and according to Coordinate conversion relation calculates the corresponding fetching of crawl point in the crawl point range；
The corresponding position coordinates of each crawl point are extracted from the fetching, and according to the position coordinates, are determined Crawl point on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance, by identified crawl point Corresponding fetching alternately fetching；
Extract the corresponding deflection angle of each crawl point from the alternative fetching, and according to the deflection angle, described Centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching；
Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching.
As it can be seen that electronic equipment obtains the point cloud data of object to be grabbed, root in the scheme provided by the embodiment of the present invention The centroid position that object to be grabbed is calculated according to point cloud data, according to preset rules, determines to include centroid position according to centroid position Crawl point range the crawl point in crawl point range is coordinately transformed then according to default crawl point-generating algorithm, and The corresponding fetching of crawl point in crawl point range is calculated, then extracts the corresponding position of each crawl point from fetching Coordinate, and according to the position coordinates, determine on preset coordinate direction with mechanical arm tail end current location first it is default away from From interior crawl point, by the corresponding fetching of identified crawl point alternately fetching, finally from alternative crawl side The corresponding deflection angle of each crawl point is extracted in case, and according to deflection angle, centroid position and mechanical arm tail end current location, is calculated The corresponding alternative fetching in the smallest axial direction drift angle is determined as target and grabbed by each corresponding axial drift angle of alternative fetching Take scheme.Since target fetching is the process to the crawl point in crawl point range after three-dimensional space is coordinately transformed Determination is screened in distance screening and axial drift angle, therefore the crawl to any attitude, any position object may be implemented, crawl effect Rate and flexibility are largely increased.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete Door or transistor logic, discrete hardware components.
Wherein, the step of above-mentioned point cloud data for obtaining object to be grabbed may include:
Obtain the color image and depth image of the object to be grabbed；
The color image is inputted into preset convolutional neural networks and carries out target detection, the object to be grabbed is obtained and exists Target position in the color image；
Point cloud data is converted by the pixel in the depth image with the target position corresponding part.
Wherein, above-mentioned that the crawl point model including the centroid position is determined according to preset rules according to the centroid position The step of enclosing may include:
In three-dimensional space, determine that the range at a distance from the centroid position in the second pre-determined distance is crawl point model It encloses.
Wherein, the default crawl point-generating algorithm of above-mentioned basis carries out coordinate change to the crawl point in the crawl point range The step of changing, and according to coordinate conversion relation, calculating the crawl point corresponding fetching in the crawl point range, can wrap It includes:
According to Generate grasp pose candidates algorithm, to the crawl in the crawl point range The local coordinate system of point carries out rotation transformation；
It is sat according to the rotation transformation relationship of the local coordinate system and the mechanical arm coordinate system and the point cloud data The projection relation of system is marked, the corresponding fetching of each crawl point in the crawl point range is calculated.
Wherein, above-mentioned that the corresponding deflection angle of each crawl point is extracted from the alternative fetching, and according to described inclined Corner, the centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching The step of, may include:
The corresponding deflection angle of each crawl point is extracted from the alternative fetching；
According to the centroid position and the mechanical arm tail end current location, calculate the mechanical arm current location relative to The deflection angle of the object to be grabbed；
By the difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point, it is determined as each crawl The corresponding axial drift angle of point.
Wherein, above-mentioned centroid position can be expressed as (x0, y0), the mechanical arm tail end current location can be expressed as (x*, y*)；
It is above-mentioned according to the centroid position and the mechanical arm tail end current location, calculate mechanical arm current location phase For the object to be grabbed deflection angle the step of, may include:
According to formulaThe mechanical arm current location is calculated relative to the object to be grabbed Deflection angle theta.
Wherein, the above method can also include:
Control the crawl that the mechanical arm carries out object to be grabbed according to the target fetching.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer readable storage medium memory Computer program is contained, the computer program performs the steps of when being executed by processor
Obtain the point cloud data of object to be grabbed；
The centroid position of the object to be grabbed is calculated according to the point cloud data；
The crawl point range including the centroid position is determined according to preset rules according to the centroid position；
According to default crawl point-generating algorithm, the crawl point in the crawl point range is coordinately transformed, and according to Coordinate conversion relation calculates the corresponding fetching of crawl point in the crawl point range；
The corresponding position coordinates of each crawl point are extracted from the fetching, and according to the position coordinates, are determined Crawl point on preset coordinate direction with mechanical arm tail end current location in the first pre-determined distance, by identified crawl point Corresponding fetching alternately fetching；
Extract the corresponding deflection angle of each crawl point from the alternative fetching, and according to the deflection angle, described Centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching；
Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching.
As it can be seen that when computer program is executed by processor, obtaining object to be grabbed in scheme provided by the embodiment of the present invention The point cloud data of body calculates the centroid position of object to be grabbed according to point cloud data, according to centroid position, according to preset rules, The crawl point range including centroid position is determined, then according to default crawl point-generating algorithm, to the crawl in crawl point range Point is coordinately transformed, and calculates the corresponding fetching of crawl point in crawl point range, then extract often from fetching The corresponding position coordinates of a crawl point, and according to the position coordinates, determination is worked as on preset coordinate direction with mechanical arm tail end Crawl point of the front position in the first pre-determined distance, by the corresponding fetching of the identified crawl point alternately side of crawl Case finally extracts the corresponding deflection angle of each crawl point from alternative fetching, and according to deflection angle, centroid position and machinery Arm end current location calculates the corresponding axial drift angle of each alternative fetching, and the smallest axial drift angle is corresponding alternative Fetching is determined as target fetching.Since target fetching is to the crawl point in crawl point range in three-dimensional space After being coordinately transformed, determining by distance screening and axial drift angle screening, therefore may be implemented to any attitude, any position The crawl of glove body, grabs efficiency and flexibility is largely increased.
Wherein, the step of above-mentioned point cloud data for obtaining object to be grabbed may include:
Obtain the color image and depth image of the object to be grabbed；
The color image is inputted into preset convolutional neural networks and carries out target detection, the object to be grabbed is obtained and exists Target position in the color image；
Point cloud data is converted by the pixel in the depth image with the target position corresponding part.
Wherein, above-mentioned that the crawl point model including the centroid position is determined according to preset rules according to the centroid position The step of enclosing may include:
In three-dimensional space, determine that the range at a distance from the centroid position in the second pre-determined distance is crawl point model It encloses.
Wherein, the default crawl point-generating algorithm of above-mentioned basis carries out coordinate change to the crawl point in the crawl point range The step of changing, and according to coordinate conversion relation, calculating the crawl point corresponding fetching in the crawl point range, can wrap It includes:
According to Generate grasp pose candidates algorithm, to the crawl in the crawl point range The local coordinate system of point carries out rotation transformation；
It is sat according to the rotation transformation relationship of the local coordinate system and the mechanical arm coordinate system and the point cloud data The projection relation of system is marked, the corresponding fetching of each crawl point in the crawl point range is calculated.
Wherein, above-mentioned that the corresponding deflection angle of each crawl point is extracted from the alternative fetching, and according to described inclined Corner, the centroid position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching The step of, may include:
The corresponding deflection angle of each crawl point is extracted from the alternative fetching；
According to the centroid position and the mechanical arm tail end current location, calculate the mechanical arm current location relative to The deflection angle of the object to be grabbed；
By the difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point, it is determined as each crawl The corresponding axial drift angle of point.
Wherein, above-mentioned centroid position can be expressed as (x0, y0), the mechanical arm tail end current location can be expressed as (x*, y*)；
It is above-mentioned according to the centroid position and the mechanical arm tail end current location, calculate mechanical arm current location phase For the object to be grabbed deflection angle the step of, may include:
According to formulaThe mechanical arm current location is calculated relative to the object to be grabbed Deflection angle theta.
Wherein, the above method can also include:
Control the crawl that the mechanical arm carries out object to be grabbed according to the target fetching.
It should be noted that for above-mentioned apparatus, electronic equipment and computer readable storage medium embodiment, due to It is substantially similar to embodiment of the method, so being described relatively simple, related place is referring to the part explanation of embodiment of the method It can.
Need further exist for explanation, herein, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning Covering non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes that A little elements, but also including other elements that are not explicitly listed, or further include for this process, method, article or The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged Except there is also other identical elements in the process, method, article or apparatus that includes the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention It is interior.

Claims (10)

1. a kind of generation method of mechanical arm fetching, which is characterized in that the described method includes:
Obtain the point cloud data of object to be grabbed；
The centroid position of the object to be grabbed is calculated according to the point cloud data；
The crawl point range including the centroid position is determined according to preset rules according to the centroid position；
According to default crawl point-generating algorithm, the crawl point in the crawl point range is coordinately transformed, and according to coordinate Transformation relation calculates the corresponding fetching of crawl point in the crawl point range；
The corresponding position coordinates of each crawl point are extracted from the fetching, and according to the position coordinates, are determined pre- If the crawl point on coordinate direction with mechanical arm tail end current location in the first pre-determined distance, identified crawl point is corresponded to Fetching alternately fetching；
The corresponding deflection angle of each crawl point is extracted from the alternative fetching, and according to the deflection angle, the mass center Position and the mechanical arm tail end current location calculate the corresponding axial drift angle of each alternative fetching；
Alternative fetching corresponding to the smallest axial drift angle is determined as target fetching.
2. the method as described in claim 1, which is characterized in that the step of the point cloud data for obtaining object to be grabbed, packet It includes:
Obtain the color image and depth image of the object to be grabbed；
The color image is inputted into preset convolutional neural networks and carries out target detection, obtains the object to be grabbed described Target position in color image；
Point cloud data is converted by the pixel in the depth image with the target position corresponding part.
3. the method as described in claim 1, which is characterized in that described to be determined according to the centroid position according to preset rules Include the steps that the crawl point range of the centroid position, comprising:
In three-dimensional space, determine that the range at a distance from the centroid position in the second pre-determined distance is crawl point range.
4. the method as described in claim 1, which is characterized in that the default crawl point-generating algorithm of the basis, to the crawl Crawl point in point range is coordinately transformed, and according to coordinate conversion relation, calculates the crawl point in the crawl point range The step of corresponding fetching, comprising:
According to Generate grasp pose candidates algorithm, to the crawl point in the crawl point range Local coordinate system carries out rotation transformation；
According to the rotation transformation relationship of the local coordinate system and the mechanical arm coordinate system and the point cloud data coordinate system Projection relation, calculate the corresponding fetching of each crawl point in the crawl point range.
5. the method as described in claim 1, which is characterized in that described to extract each crawl point from the alternative fetching Corresponding deflection angle, and according to the deflection angle, the centroid position and the mechanical arm tail end current location, it calculates each standby The step of selecting fetching corresponding axial drift angle, comprising:
The corresponding deflection angle of each crawl point is extracted from the alternative fetching；
According to the centroid position and the mechanical arm tail end current location, the mechanical arm current location is calculated relative to described The deflection angle of object to be grabbed；
By the difference of the corresponding deflection angle of mechanical arm deflection angle corresponding with each crawl point, it is determined as each crawl point pair The axial drift angle answered.
6. method as claimed in claim 5, which is characterized in that the centroid position is expressed as (x0, y0), the mechanical arm end End current location is expressed as (x*, y*)；
It is described according to the centroid position and the mechanical arm tail end current location, calculate the mechanical arm current location relative to The step of deflection angle of the object to be grabbed, comprising:
According to formulaCalculate deflection of the mechanical arm current location relative to the object to be grabbed Angle θ.
7. as the method according to claim 1 to 6, which is characterized in that the method also includes:
Control the crawl that the mechanical arm carries out object to be grabbed according to the target fetching.
8. a kind of generating means of mechanical arm fetching, which is characterized in that described device includes:
Point cloud data obtains module, for obtaining the point cloud data of object to be grabbed；
Centroid position determining module, for calculating the centroid position of the object to be grabbed according to the point cloud data；
Point range determining module is grabbed, for according to preset rules, determining to include the centroid position according to the centroid position Crawl point range；
Fetching determining module, for being clicked through to the crawl in the crawl point range according to default crawl point-generating algorithm Row coordinate transform, and according to coordinate conversion relation, calculate the corresponding fetching of crawl point in the crawl point range；
Alternative fetching determining module, for extracting the corresponding position coordinates of each crawl point from the fetching, and According to the position coordinates, determination grabbing in the first pre-determined distance with mechanical arm tail end current location on preset coordinate direction It takes a little, by the corresponding fetching of identified crawl point alternately fetching；
Axial drift angle determining module, for extracting the corresponding deflection angle of each crawl point, and root from the alternative fetching According to the deflection angle, the centroid position and the mechanical arm tail end current location, it is corresponding to calculate each alternative fetching Axial drift angle；
Target fetching generation module is grabbed for alternative fetching corresponding to the smallest axial drift angle to be determined as target Take scheme.
9. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing Device, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes method and step as claimed in claim 1 to 7.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Program, the computer program realize method and step as claimed in claim 1 to 7 when being executed by processor.
CN201710891044.1A 2017-09-27 2017-09-27 Method and device for generating mechanical arm grabbing scheme Active CN109559341B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710891044.1A CN109559341B (en) 2017-09-27 2017-09-27 Method and device for generating mechanical arm grabbing scheme

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710891044.1A CN109559341B (en) 2017-09-27 2017-09-27 Method and device for generating mechanical arm grabbing scheme

Publications (2)

Publication Number Publication Date
CN109559341A true CN109559341A (en) 2019-04-02
CN109559341B CN109559341B (en) 2021-03-26

Family

ID=65863718

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710891044.1A Active CN109559341B (en) 2017-09-27 2017-09-27 Method and device for generating mechanical arm grabbing scheme

Country Status (1)

Country Link
CN (1) CN109559341B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110181500A (en) * 2019-06-06 2019-08-30 广东海洋大学 A kind of control system and its control method of bionic mechanical hand
CN110744544A (en) * 2019-10-31 2020-02-04 昆山市工研院智能制造技术有限公司 Service robot vision grabbing method and service robot
CN113319848A (en) * 2020-02-28 2021-08-31 精工爱普生株式会社 Robot control method and robot system
CN113674348A (en) * 2021-05-28 2021-11-19 中国科学院自动化研究所 Object grabbing method, device and system
CN113731860A (en) * 2021-09-03 2021-12-03 西安建筑科技大学 Automatic sorting system and method for objects stacked in container
CN115213721A (en) * 2022-09-21 2022-10-21 江苏友邦精工实业有限公司 A upset location manipulator for automobile frame processing
CN115995013A (en) * 2023-03-21 2023-04-21 江苏金恒信息科技股份有限公司 Covering agent adding method, covering agent adding device, computer equipment and storage medium
EP4044125A4 (en) * 2020-10-20 2023-08-02 Samsung Electronics Co., Ltd. Electronic apparatus and control method thereof

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4059614B2 (en) * 2000-05-22 2008-03-12 三菱電機株式会社 Control device for 3D laser processing machine
CN104217441A (en) * 2013-08-28 2014-12-17 北京嘉恒中自图像技术有限公司 Mechanical arm positioning fetching method based on machine vision
CN106530297A (en) * 2016-11-11 2017-03-22 北京睿思奥图智能科技有限公司 Object grabbing region positioning method based on point cloud registering
CN106874914A (en) * 2017-01-12 2017-06-20 华南理工大学 A kind of industrial machinery arm visual spatial attention method based on depth convolutional neural networks
CN106932780A (en) * 2017-03-14 2017-07-07 北京京东尚科信息技术有限公司 Object positioning method, device and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4059614B2 (en) * 2000-05-22 2008-03-12 三菱電機株式会社 Control device for 3D laser processing machine
CN104217441A (en) * 2013-08-28 2014-12-17 北京嘉恒中自图像技术有限公司 Mechanical arm positioning fetching method based on machine vision
CN106530297A (en) * 2016-11-11 2017-03-22 北京睿思奥图智能科技有限公司 Object grabbing region positioning method based on point cloud registering
CN106874914A (en) * 2017-01-12 2017-06-20 华南理工大学 A kind of industrial machinery arm visual spatial attention method based on depth convolutional neural networks
CN106932780A (en) * 2017-03-14 2017-07-07 北京京东尚科信息技术有限公司 Object positioning method, device and system

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110181500A (en) * 2019-06-06 2019-08-30 广东海洋大学 A kind of control system and its control method of bionic mechanical hand
CN110744544A (en) * 2019-10-31 2020-02-04 昆山市工研院智能制造技术有限公司 Service robot vision grabbing method and service robot
CN113319848A (en) * 2020-02-28 2021-08-31 精工爱普生株式会社 Robot control method and robot system
CN113319848B (en) * 2020-02-28 2023-11-14 精工爱普生株式会社 Robot control method and robot system
EP4044125A4 (en) * 2020-10-20 2023-08-02 Samsung Electronics Co., Ltd. Electronic apparatus and control method thereof
CN113674348A (en) * 2021-05-28 2021-11-19 中国科学院自动化研究所 Object grabbing method, device and system
CN113674348B (en) * 2021-05-28 2024-03-15 中国科学院自动化研究所 Object grabbing method, device and system
CN113731860A (en) * 2021-09-03 2021-12-03 西安建筑科技大学 Automatic sorting system and method for objects stacked in container
CN113731860B (en) * 2021-09-03 2023-10-24 西安建筑科技大学 Automatic sorting system and method for piled articles in container
CN115213721A (en) * 2022-09-21 2022-10-21 江苏友邦精工实业有限公司 A upset location manipulator for automobile frame processing
CN115995013A (en) * 2023-03-21 2023-04-21 江苏金恒信息科技股份有限公司 Covering agent adding method, covering agent adding device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN109559341B (en) 2021-03-26

Similar Documents

Publication Publication Date Title
CN109559341A (en) A kind of generation method and device of mechanical arm fetching
CN111738261B (en) Single-image robot unordered target grabbing method based on pose estimation and correction
CN110298370A (en) Network model training method, device and object pose determine method, apparatus
CN108555908A (en) A kind of identification of stacking workpiece posture and pick-up method based on RGBD cameras
JP2019056966A (en) Information processing device, image recognition method and image recognition program
CN107813313A (en) The bearing calibration of manipulator motion and device
CN107395958B (en) Image processing method and device, electronic equipment and storage medium
CN110211180A (en) A kind of autonomous grasping means of mechanical arm based on deep learning
CN109816730A (en) Workpiece grabbing method, apparatus, computer equipment and storage medium
CN107953329A (en) Object identification and Attitude estimation method, apparatus and mechanical arm grasping system
CN107571260A (en) The method and apparatus that control machine people captures object
CN111144349B (en) Indoor visual relocation method and system
CN108765489A (en) A kind of pose computational methods, system, medium and equipment based on combination target
CN110400315A (en) A kind of defect inspection method, apparatus and system
CN109064400A (en) Three-dimensional point cloud joining method, apparatus and system
CN109909998A (en) A kind of method and device controlling manipulator motion
CN109961471A (en) A kind of mask method, device and the electronic equipment of objects in images position
CN107452028B (en) Method and device for determining position information of target image
CN110293553A (en) Control the method, apparatus and model training method, device of robotic arm manipulation object
CN112950667A (en) Video annotation method, device, equipment and computer readable storage medium
CN113329179B (en) Shooting alignment method, device, equipment and storage medium
CN110298877A (en) A kind of the determination method, apparatus and electronic equipment of object dimensional pose
CN109670501A (en) Object identification and crawl position detection method based on depth convolutional neural networks
CN112950528A (en) Certificate posture determining method, model training method, device, server and medium
CN109978956A (en) Acquire scaling method, device and the calibration system of equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant