CN110433467A - Picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm - Google Patents
Picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm Download PDFInfo
- Publication number
- CN110433467A CN110433467A CN201910744905.2A CN201910744905A CN110433467A CN 110433467 A CN110433467 A CN 110433467A CN 201910744905 A CN201910744905 A CN 201910744905A CN 110433467 A CN110433467 A CN 110433467A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- table tennis
- robot
- picking
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 24
- 230000009471 action Effects 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 5
- 238000003860 storage Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 12
- 239000003016 pheromone Substances 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 241000257303 Hymenoptera Species 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000008020 evaporation Effects 0.000 description 1
- 238000001704 evaporation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 235000001968 nicotinic acid Nutrition 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B47/00—Devices for handling or treating balls, e.g. for holding or carrying balls
- A63B47/02—Devices for handling or treating balls, e.g. for holding or carrying balls for picking-up or collecting
- A63B47/021—Devices for handling or treating balls, e.g. for holding or carrying balls for picking-up or collecting for picking-up automatically, e.g. by apparatus moving over the playing surface
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B67/00—Sporting games or accessories therefor, not provided for in groups A63B1/00 - A63B65/00
- A63B67/04—Table games physically beneficial for the human body, modelled on outdoor sports, e.g. table tennis
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physical Education & Sports Medicine (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm, method include: the RGB image for obtaining binocular camera shooting, and table tennis is detected from RGB image and obtains the coordinate position of each table tennis；The depth data for obtaining binocular camera, obtains distance of each table tennis apart from video camera；The coordinate point set of the table tennis under global coordinate system is calculated；Most short traverse path is carried out using ant group algorithm to coordinate point set to plan, obtains preliminary picking up course of action diameter；The table tennis repeatedly passed through is removed from preliminary course of action diameter of picking up, obtains the path of no redundancy route.The present invention realizes the detection and positioning of table tennis, convenience and high-efficiency using binocular camera simultaneously, while ensure that precision, so that it is more intelligent to pick up ball process；Redundancy route optimization is carried out to the path that ant group algorithm is cooked up, the walking path of robot is further reduced, has saved the time.
Description
Technical field
The present invention relates to robotic technology fields, and in particular to a kind of picking up table tennis ball based on binocular vision and ant group algorithm
Robot operation method and equipment.
Background technique
With artificial intelligence technology and Chinese society expanding economy, services humanoid robot and gradually come into public family
Front yard.Table tennis is deep to be liked by the people as national ball game.But it following largely pick up ball work and but allows people very tired
It is angry, there is special messenger to be responsible for picking up ball in table tennis clubs at present, human cost is higher.And on the market without it is a can popularize pick up
Ping-pong robot, only research also bias toward academic research, and actual engineer application is seldom, and mostly there was only scattered side
Method.
Existing detection and location technology complexity are cumbersome, are difficult to complete to determine table tennis using common RGB camera
Position/task, therefore most of ball picking robots can not position ball, thus cannot achieve the global traversal road to all balls
Diameter planning, air navigation aid are very inefficient.
Ant group algorithm is a kind of bionics algorithm, is mainly used for TSP problem, it can acquire one within a very short time and approach
Optimal solution.But its practical application is limited when on ball picking robot by robot width, needs to carry out further excellent
Change.
Summary of the invention
It is an object of the invention to overcome deficiency in the prior art, propose a kind of based on binocular vision and ant group algorithm
Picking up table tennis ball robot operation method and equipment, realize the detection and positioning of table tennis, and carry out to the path cooked up
Redundancy route optimization further reduces the walking path of robot, has saved the time.
In order to solve the above technical problems, the present invention provides a kind of picking up table tennis ball machine based on binocular vision and ant group algorithm
Device people's operation method, characterized in that including following procedure:
Establish the global coordinate system in table tennis training place；
It obtains ground table tennis in the place of binocular camera shooting to be scattered the RGB image of situation, be detected from RGB image
Table tennis and obtain the coordinate position of each table tennis out；
The depth data for obtaining binocular camera, obtains distance of each table tennis apart from video camera；
According to the coordinate position of each table tennis and apart from the distance of video camera, the table tennis under global coordinate system is calculated
Coordinate point set；
Most short traverse path is carried out using ant group algorithm to coordinate point set to plan, obtains preliminary picking up course of action diameter；
The table tennis repeatedly passed through is removed from preliminary course of action diameter of picking up, obtains the path of no redundancy route.
Further, table tennis is detected from RGB image and obtains the detailed process packet of the coordinate position of each table tennis
It includes:
RGB image is converted into HSV image；
HSV image is changed into grayscale image, carries out Gaussian Blur；
Contour detecting is carried out using canny operator, obtains the profile of all objects in figure；
Using the circle in hough-circle transform detection figure, the table tennis in place can be detected out, and all table tennis are calculated
Coordinate position of pang the ball in figure.
Further, the distance according to the coordinate position of each table tennis and apart from video camera is calculated in global coordinate system
The coordinate point set of lower table tennis includes following procedure:
By the coordinate position of each table tennis and apart from the distance of video camera, it is transformed under camera three-dimensional system of coordinate, obtains
The coordinate point set of table tennis under camera measurements of the chest, waist and hips coordinate system；
Again by table tennis in the case where the coordinate under camera three-dimensional system of coordinate is transformed into robot coordinate system；
Finally table tennis is obtained in the case where the coordinate under robot coordinate system is transformed into global coordinate system in global coordinate system
The coordinate point set of lower table tennis.
Further, the coordinate position of each table tennis and distance apart from video camera are transformed under camera three-dimensional system of coordinate
Process include:
Establish camera three-dimensional system of coordinate XYZ, plane of delineation coordinate system xy；
The mapping relations of both camera measurements of the chest, waist and hips coordinate system midpoint P (X, Y, Z) and plane of delineation coordinate system midpoint p (x, y) are as follows:
Wherein, f is camera focus, and Z is the depth of the table tennis obtained from depth data, and x indicates plane of delineation coordinate
The x-axis coordinate of system, X indicate the X axis coordinate in camera three-dimensional system of coordinate；Y indicates the y-axis coordinate in plane of delineation coordinate system, Y table
Show the Y axis coordinate in camera three-dimensional system of coordinate；cxThe coordinate for being principal point in x-axis, cyFor the coordinate of principal point on the y axis, fxFor f
Pixel dimension in x-axis, fyFor the pixel dimension of f on the y axis；
The coordinate position of each table tennis is coordinate in plane of delineation coordinate system, and the distance apart from video camera is camera coordinates system
Three-dimensional coordinate P (X, Y, Z) of the table tennis in camera coordinates system is calculated according to above-mentioned formula in middle Z axis numerical value.
Further, table tennis is transformed into the process under global coordinate system in the coordinate under robot coordinate system are as follows:
Robot coordinate ties up to the origin of global coordinate system when starting, and coordinate is (x0, y0, th0)=(0,0,0), if machine
People movement speed V=(Vx, Vy, Vth), VxRefer to speed along the x-axis direction, VyRefer to speed along the y-axis direction, VthRefer to robot rotation
Rotary speed；Then after the dt time, coordinate of the robot coordinate system relative to world coordinates origin are as follows:
(x1, y1, th1)=(x0+(Vx*cos(th0)-Vy*sin(th0)) * dt, y0+(Vx*sin(th0)+Vy*cos
(th0)) * dt, th0+Vth*dt) (5)
According to the coordinate transformation relation for the robot coordinate system that formula (5) obtains, coordinate conversion can be carried out, in robot
Target table tennis spherical coordinates (X ', Y ') under coordinate system is transformed into the coordinate (X ", Y ") under global coordinate system are as follows:
(X ", Y ")=(x1+(X′*cos(th1)-Y′*sin(th1)),
y1+(X′*sin(th1)+Y′*cos(th1)))
The coordinate conversion for completing all tablees tennis, obtains the coordinate point set of the table tennis under global coordinate system.
Further, the table tennis for removing in course of action diameter and repeatedly passing through is picked up from preliminary, obtains the road of no redundancy route
Diameter includes following procedure:
Course of action diameter is picked up according to preliminary, it is assumed that robot is mobile to be moved to the 1st target table tennis from starting point,
Draw straight line b1、b2With d1、d2, b1、b2It is parallel to the line of starting point point and target table tennis two o'clock, and and machine
Device people is tangent in the minimum circumscribed circle of starting point；Straight line d1It is vertical with two o'clock line direction and and robot starting point minimum
The tail portion of circumscribed circle is tangent；Straight line d2It is vertical with two o'clock line direction and and minimum circumscribed circle of the robot in target point head
It is tangent；The rectangle that this four lines are constituted indicates robot region inswept from starting point to target point, calculates and obtains in four lines
All table tennis coordinate points in the rectangle of composition leave out these points from coordinate points concentration,
Then using this target point as starting point, next target point is chosen in course of action diameter from preliminary picking up, is carried out above-mentioned
Same operation；And so on, until the point for picking up all on course of action diameter is all disposed, finally obtaining one does not have redundancy road
The path of line.
Correspondingly, the present invention also provides a kind of computer readable storage medium for storing one or more programs, it is described
One or more programs include instruction, described instruction when executed by a computing apparatus so that the above-mentioned side of calculatings equipment execution
Method.
Correspondingly, the present invention also provides a kind of calculating equipment, including,
One or more processors, memory and one or more programs, wherein one or more programs are stored in institute
It states in memory and is configured as being executed by one or more of processors, one or more of programs include for executing
The instruction of the above method.
Compared with prior art, the beneficial effects obtained by the present invention are as follows being: the present invention is using binocular camera while realizing
The detection and positioning of table tennis, convenience and high-efficiency, while ensure that precision, so that it is more intelligent to pick up ball process；Ant colony is calculated
The path that method is cooked up carries out redundancy route optimization, further reduces the walking path of robot, has saved the time.
Detailed description of the invention
Fig. 1 is the flow diagram of method of the invention；
Fig. 2 is the flow diagram of table tennis detection and positioning；
Fig. 3 is camera national forest park in Xiaokeng schematic diagram；
Fig. 4 is base_link and map coordinate system transition diagram；
Fig. 5 is the flow diagram that ant group algorithm is used for traverse path planning；
Fig. 6 is redundant points schematic diagram.
Specific embodiment
The invention will be further described below in conjunction with the accompanying drawings.Following embodiment is only used for clearly illustrating the present invention
Technical solution, and not intended to limit the protection scope of the present invention.
A kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm of the invention, referring to Fig. 1 institute
Show, comprising the following steps:
Step 1, the map for obtaining table tennis training place, establishes global coordinate system.
Table tennis is obtained using the gmapping graph builder under laser radar and ROS (robot operating system) platform first
The static map of pang ball training court, then to map is denoised and is modified, and obtains available global static grating map, machine
Device people carries out navigation avoidance according to this map.As global coordinate system map, this coordinate system with rectangle training court some
Angle point is as origin, along the length-width direction of training court respectively as transverse and longitudinal coordinate axis.The table tennis detected below will
It is transformed under this coordinate system and shows.
Step 2, the RGB image for obtaining binocular camera, from detecting table tennis in RGB image and obtain it in figure
Coordinate points.
Binocular camera uses kinect, it has a RGB color camera and a depth camera.Binocular camera
It is mounted on robot front center position, visual angle head-up front takes 15cm to 40cm to be advisable apart from ground level.Robot is being picked up
Ball task start before shoot ground table tennis, in ROS platform simultaneously receive synchronization kinect RGB data and
Depth data guarantees that two data time stamps obtained are consistent.
As shown in Fig. 2, taking out the RGB image comprising several tablees tennis in the sometime place of kinect shooting, and will
Image is converted to HSV image, is filtered by the HSV numerical threshold of white table tennis, takes out white portion in image, that is, takes out
White object image including table tennis.Then HSV image is changed into grayscale image, carries out Gaussian Blur, allow image border
It is smoother.It reuses canny operator and carries out contour detecting, setting threshold value is 100 to 200, takes out the profile of all objects in figure.
Finally using the circle in hough-circle transform detection figure, the table tennis (coordinate of the table tennis in figure) in place can be detected out,
And the table tennis that coordinate point set of all tablees tennis in figure is calculated, and these are detected according to the sequence of detection from 1
It is numbered to N (table tennis number).
Step 3, the depth data for obtaining binocular camera obtains its corresponding depth according to the two-dimensional coordinate of table tennis and believes
It ceases (i.e. horizontal distance of the table tennis apart from camera), table tennis is calculated under robot coordinate system according to the depth information
Coordinate.
According to camera pinhole model, i.e., Utopian national forest park in Xiaokeng obtains two coordinate systems, and one as shown in Figure 3
Camera three-dimensional system of coordinate XYZ, wherein X-axis is parallel to ground and is parallel to the plane of delineation, Y-axis perpendicular to ground, Z axis perpendicular to
The plane of delineation；One is plane of delineation coordinate system xy, and the plane of delineation is perpendicular to ground；According to the intercept theorem in geometry, obtain
The mapping relations of both camera measurements of the chest, waist and hips coordinate system midpoint P (X, Y, Z) and plane of delineation coordinate system midpoint p (x, y) are as follows:
Wherein, f is camera focus, and Z is the depth of the table tennis obtained from depth data, and x indicates plane of delineation coordinate
The x-axis coordinate of system, X indicate the X axis coordinate in camera three-dimensional system of coordinate；Y indicates the y-axis coordinate in plane of delineation coordinate system, Y table
Show the Y axis coordinate in camera three-dimensional system of coordinate.
According to camera internal reference, above step is improved:
Due to actual difference, the principal point (origin of plane of delineation coordinate system) of each camera is different to establish a capital in image
Center；The pixel in the direction x and y is also variant, therefore above-mentioned formula (1) (2) are converted are as follows:
Wherein, cxThe coordinate for being principal point in x-axis, cyFor the coordinate of principal point on the y axis, fxThe amount of pixels for being f in x-axis
Guiding principle, fyFor the pixel dimension of f on the y axis.
As shown in flow chart 2.The internal reference matrix of calling interface function acquisition camera first, then according to above-mentioned formula (3)
(4) three-dimensional coordinate P (X, Y, Z) of the target table tennis in camera coordinates system is calculated, and (X, Y are obtained by calculated above, and Z is
The depth of the point, that is, horizontal distance of the table tennis apart from camera).Due to camera three-dimensional system of coordinate and robot coordinate system
(entitled base_link, using robot direction of advance as X-axis, the left side is Y-axis) direction is inconsistent, there are transformational relation, final table tennis
Coordinate of pang the ball under robot coordinate system is that (assuming that table tennis is whole on the ground, height coordinate is (X ', Y ')=(Z, X)
0) display in the RVIZ (ROS visualization tool), and under ROS platform.
Above procedure is repeated, the coordinate under robot coordinate system of table tennis composition all on ground in image is obtained
Point set.
Step 4: according to the transformational relation of robot coordinate system and global coordinate system, to the table tennis under robot coordinate system
Coordinate conversion is carried out, coordinate point set of the table tennis under global coordinate system is obtained.
The entitled map of global coordinate system, the entitled base_link of robot coordinate system.Base_link coordinate system exists when starting
(coordinate is (x to the origin of map coordinate system0, y0, th0)=(0,0,0), x, y, th respectively refer to x-axis coordinate, y-axis coordinate, two seats
The angle of mark system offset), if robot movement speed V=(Vx, Vy, Vth), VxRefer to speed along the x-axis direction, VyRefer to along y-axis
The speed in direction, VthRefer to robot rotation speed.Then after the dt time, base_link coordinate system is relative to world coordinates original
The coordinate of point are as follows:
(x1, y1, th1)=(x0+(Vx*cos(th0)-Vy*sin(th0)) * dt, y0+(Vx*sin(th0)+Vy*cos
(th0)) * dt, th0+Vth*dt) (5)
According to the coordinate transformation relation for the base_link coordinate system that formula (5) obtains, coordinate conversion can be carried out.By
Origin (the x of base_link coordinate system1, y1) and deflection angle th1The coordinate under global coordinate system can be calculated (by Fig. 4 institute
Show).The seat under global coordinate system map is transformed into target table tennis spherical coordinates (X ', Y ') under robot coordinate system base_link
It marks (X ", Y ") are as follows:
(X ", Y ")=(x1+(X′*cos(th1)-Y′*sin(th1)),
y1+(X′*sin(th1)+Y′*cos(th1)))
So far whole coordinate conversion works is completed.
Step 5: most short traverse path being carried out using ant group algorithm to obtained table tennis coordinate point set and is planned, is obtained preliminary
Pick up ball sequence.
Number order when first, in accordance with detection table tennis, calculates all tablees tennis distance between any two and is combined into two
Adjacency matrix Graph is tieed up, matrix row and column dimension is all the table tennis number N detected in figure, for carrying out shortest path
It calculates.Element in matrix is the distance between table tennis, such as row i and column j indicate between i-th of ball and j-th of ball away from
From.There is N number of table tennis, then the number of element is N in matrix2。
As shown in figure 5, the parameter alpha (weighted values of pheromones) of elder generation initialization with Ant colony algorithm when starting, β (visibility plus
Weight), ρ (evaporation raties of pheromones), Q (pheromones gain coefficient), initialization information element τ0, the number of iterations N.
Then table tennis is randomly choosed as starting point for each ant, and starting point is added to and is had already passed through
Point set C.Calculate separately current point to it is remaining without all the points (set O) probability P=τα*(γ)βWherein τ is two o'clock
Between pheromones pheromones, γ be distance between two points (distance is obtained from adjacency matrix graph) inverse.Using roulette
Method, select next point for each ant.A random number q is taken, the probability to first point is calculated, if it is less than q
Then continue to calculate next point；If the probability to some point is cumulative and is greater than q, the point is taken.For example, q=0.4, p1=
0.2, p2=0.3, p3=0.5, (pnIndicate the probability of n-th point of selection), then due to p1< 0.4, and p1+p2> 0.4, so taking
Second point is as next target.And so on all ants are moved to next target point, and the point is added to
In set through passing through (set C).
Above step is repeated, until all points are all added in the set having already passed through.Thus complete an iteration.
According to the path length that every ant of path computing of selection is passed by, i.e., according to path order, calculate table tennis it
Between distance and cumulative summation, such as path order is 2-1-3-4 (serial number in front demarcated by step), then path length is
Distance d of No. 2 tablees tennis to No. 1 table tennis1, in addition distance d of No. 1 ball to No. 3 balls2, in addition distance of No. 3 balls to No. four balls
d3, i.e. L=d1+d2+d3.All ants update the pheromones on each side: the information in each path between two o'clock after covering
Plain increment and the length in path where them are inversely proportional: add=Q/Lk(Q generally takes 50-100), wherein LkIndicate kth paths
Length, add indicate pheromones incrementss.Therefore updated pheromones are τ=(1- ρ) * τ0+Q/Lk。
Continue next round iteration using new pheromones；If (1) it is all complete to reach setting value N (50 times) for the number of iterations
At, or (2) pheromones increment add be less than certain threshold value (such as 0.01), then export traversal all the points approximate optimal solution (path
Length is most short).
Step 6: according to robot own form and the path come is cooked up, the table tennis that repeatedly passes through of removal, further
Shorten path length.
As shown in Figure 6 (in global coordinate system map), the sequence obtained according to step 5 path planning, it is assumed that robot is mobile
The 1st target table tennis be that (digital number indicates machine according to the table tennis sequential calibration identified in step 2, No. 0 point for No. 2 points
Current coordinate point of the device people when identifying table tennis, No. 2 points are the 2nd (number has been demarcated when detecting table tennis) identification
The coordinate points of table tennis out, it is used as the 1st target point in path planning), draw straight line b1、b2With d1、d2, b1、b2It is flat
Row in the line of No. 0 point and No. 2 point two o'clocks, and and robot No. 0 point minimum circumscribed circle (since robot shape may
Irregularly, robot can be enclosed in internal smallest circle completely and calls minimum circumscribed circle) it is tangent；Straight line d1With two o'clock line
Direction it is vertical and and robot it is tangent in the tail portion of the minimum circumscribed circle of starting point；Straight line d2It is vertical with two o'clock line direction and with
Robot is tangent on the head of the minimum circumscribed circle of No. 2 points；The rectangle that this four lines are constituted indicates robot from No. 0 o'clock to No. 2
The region that point is inswept (the considerations of for efficiency ignores for the unlapped part in four angles of rectangle).It calculates and obtains
All table tennis coordinate points in rectangle that four lines are constituted, i.e. point 4,7,15 in Fig. 4 delete these points from coordinate points concentration
It goes, because robot carries suction device or idler wheel ball picking up device, robot is in the process for being moved to No. 2 points from 0 point
In, these tablees tennis can be passed through, they all be picked up, i.e., machine artificially can will be within the scope of minimum circumscribed circle radial width
Ball, which is all picked up, to be come.It deletes these points and just reduces redundant traversal point, improve operational efficiency.
Then using No. 2 points as starting point, next target point is chosen from the path planning of step 5, to the two points
It is similarly operated according to the operating process of No. 0 point and No. 2 points.And so on, step 6 is repeated, until calculating in step 5
To path on all point be all disposed, finally obtain a path without redundancy route, this road is pressed by robot
Diameter pick up the efficiency of ball work than only using the path operation in step 5 efficiently, saves the time.
Correspondingly, the present invention also provides a kind of computer readable storage medium for storing one or more programs, it is described
One or more programs include instruction, described instruction when executed by a computing apparatus so that the above-mentioned side of calculatings equipment execution
Method.
Correspondingly, the present invention also provides a kind of calculating equipment, including,
One or more processors, memory and one or more programs, wherein one or more programs are stored in institute
It states in memory and is configured as being executed by one or more of processors, one or more of programs include for executing
The instruction of the above method.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, without departing from the technical principles of the invention, several improvements and modifications, these improvements and modifications can also be made
Also it should be regarded as protection scope of the present invention.
Claims (8)
1. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm, characterized in that including following mistake
Journey:
Establish the global coordinate system in table tennis training place；
It obtains ground table tennis in the place of binocular camera shooting to be scattered the RGB image of situation, table tennis is detected from RGB image
Pang ball and the coordinate position for obtaining each table tennis；
The depth data for obtaining binocular camera, obtains distance of each table tennis apart from video camera；
According to the coordinate position of each table tennis and apart from the distance of video camera, the seat of the table tennis under global coordinate system is calculated
Punctuate collection；
Most short traverse path is carried out using ant group algorithm to coordinate point set to plan, obtains preliminary picking up course of action diameter；
The table tennis repeatedly passed through is removed from preliminary course of action diameter of picking up, obtains the path of no redundancy route.
2. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm according to claim 1,
It is characterized in that detecting table tennis from RGB image and the detailed process for obtaining the coordinate position of each table tennis includes:
RGB image is converted into HSV image；
HSV image is changed into grayscale image, carries out Gaussian Blur；
Contour detecting is carried out using canny operator, obtains the profile of all objects in figure；
Using the circle in hough-circle transform detection figure, the table tennis in place can be detected out, and all tablees tennis are calculated
Coordinate position in figure.
3. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm according to claim 1,
It is characterized in that the table tennis under global coordinate system is calculated in the distance according to the coordinate position of each table tennis and apart from video camera
Coordinate point set include following procedure:
By the coordinate position of each table tennis and apart from the distance of video camera, it is transformed under camera three-dimensional system of coordinate, obtains in camera
The coordinate point set of table tennis under measurements of the chest, waist and hips coordinate system；
Again by table tennis in the case where the coordinate under camera three-dimensional system of coordinate is transformed into robot coordinate system；
Finally by table tennis in the case where the coordinate under robot coordinate system is transformed into global coordinate system, the table tennis under global coordinate system is obtained
The coordinate point set of pang ball.
4. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm according to claim 3,
It is characterized in that the coordinate position of each table tennis and distance apart from video camera are transformed into the process packet under camera three-dimensional system of coordinate
It includes:
Establish camera three-dimensional system of coordinate XYZ, plane of delineation coordinate system xy；
The mapping relations of both camera measurements of the chest, waist and hips coordinate system midpoint P (X, Y, Z) and plane of delineation coordinate system midpoint p (x, y) are as follows:
Wherein, f is camera focus, and Z is the depth of the table tennis obtained from depth data, and x indicates the x of plane of delineation coordinate system
Axial coordinate, X indicate the X axis coordinate in camera three-dimensional system of coordinate；Y indicates the y-axis coordinate in plane of delineation coordinate system, and Y indicates phase
Y axis coordinate in machine three-dimensional system of coordinate；cxThe coordinate for being principal point in x-axis, cyFor the coordinate of principal point on the y axis, fxIt is f in x-axis
On pixel dimension, fyFor the pixel dimension of f on the y axis；
The coordinate position of each table tennis is coordinate in plane of delineation coordinate system, and the distance apart from video camera is Z in camera coordinates system
Three-dimensional coordinate P (X, Y, Z) of the table tennis in camera coordinates system is calculated according to above-mentioned formula in axis values.
5. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm according to claim 3,
It is characterized in that table tennis is transformed into the process under global coordinate system in the coordinate under robot coordinate system are as follows:
Robot coordinate ties up to the origin of global coordinate system when starting, and coordinate is (x0, y0, th0)=(0,0,0), if robot moves
Dynamic speed V=(Vx, Vy, Vth), VxRefer to speed along the x-axis direction, VyRefer to speed along the y-axis direction, VthRefer to robot rotation speed
Degree；Then after the dt time, coordinate of the robot coordinate system relative to world coordinates origin are as follows:
(x1, y1, th1)=(x0+(Vx*cos(th0)-Vy*sin(th0)) * dt, y0+(Vx*sin(th0)+Vy*cos(th0)) * dt,
th0+Vth*dt) (5)
According to the coordinate transformation relation for the robot coordinate system that formula (5) obtains, coordinate conversion can be carried out, in robot coordinate
Target table tennis spherical coordinates (X ', Y ') under system is transformed into the coordinate (X ", Y ") under global coordinate system are as follows:
(X ", Y ")=(x1+(X′*cos(th1)-Y′*sin(th1)),
y1+(X′*sin(th1)+Y′*cos(th1)))
The coordinate conversion for completing all tablees tennis, obtains the coordinate point set of the table tennis under global coordinate system.
6. a kind of picking up table tennis ball robot operation method based on binocular vision and ant group algorithm according to claim 1,
It is characterized in that pick up the table tennis for removing in course of action diameter and repeatedly passing through from preliminary, obtain no redundancy route path include with
Lower process:
Course of action diameter is picked up according to preliminary, it is assumed that robot is mobile to be moved to the 1st target table tennis from starting point,
Draw straight line b1、b2With d1、d2, b1、b2It is parallel to the line of starting point point and target table tennis two o'clock, and and robot
It is tangent in the minimum circumscribed circle of starting point；Straight line d1It is vertical with two o'clock line direction and and robot starting point minimum it is external
Round tail portion is tangent；Straight line d2It is vertical with two o'clock line direction and and minimum circumscribed circle of the robot in target point head phase
It cuts；The rectangle that this four lines are constituted indicates robot region inswept from starting point to target point, calculates and obtains in four line structures
At rectangle in all table tennis coordinate points, by these points from coordinate points concentration leave out,
Then using this target point as starting point, next target point is chosen in course of action diameter from preliminary picking up, and is carried out above-mentioned same
Operation；And so on, until the point for picking up all on course of action diameter is all disposed, finally obtaining one does not have redundancy route
Path.
7. a kind of computer readable storage medium for storing one or more programs, one or more of programs include instruction,
Described instruction when executed by a computing apparatus so that the calculating equipment perform claim requires the described one kind of any one of 1-6 to be based on
The picking up table tennis ball robot operation method of binocular vision and ant group algorithm.
8. a kind of calculating equipment, including,
One or more processors, memory and one or more programs, wherein one or more programs are stored in described deposit
It in reservoir and is configured as being executed by one or more of processors, one or more of programs include being used for perform claim
It is required that a kind of any one of 1-6 instruction of the picking up table tennis ball robot operation method based on binocular vision and ant group algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910744905.2A CN110433467B (en) | 2019-08-13 | 2019-08-13 | Operation method and device of table tennis ball picking robot based on binocular vision and ant colony algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910744905.2A CN110433467B (en) | 2019-08-13 | 2019-08-13 | Operation method and device of table tennis ball picking robot based on binocular vision and ant colony algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110433467A true CN110433467A (en) | 2019-11-12 |
| CN110433467B CN110433467B (en) | 2021-02-12 |
Family
ID=68435030
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910744905.2A Active CN110433467B (en) | 2019-08-13 | 2019-08-13 | Operation method and device of table tennis ball picking robot based on binocular vision and ant colony algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110433467B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111530041A (en) * | 2020-04-24 | 2020-08-14 | 贵州民族大学 | Robot ball picking method and system |
| CN111744161A (en) * | 2020-07-29 | 2020-10-09 | 哈尔滨理工大学 | Table tennis falling point detection and edge ball wiping judgment system |
| CN112587884A (en) * | 2020-12-24 | 2021-04-02 | 烟台南山学院 | Full-automatic ball picking robot ball picking method based on monocular vision |
| CN114879669A (en) * | 2022-04-28 | 2022-08-09 | 贵州民族大学 | Robot control method based on multi-target path planning and intelligent picking |
| CN115421522A (en) * | 2022-06-20 | 2022-12-02 | 南京信息工程大学 | Unmanned aerial vehicle coverage path planning method based on parallel self-adaptive ant colony algorithm |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160105647A1 (en) * | 2011-03-25 | 2016-04-14 | Eric D. Schwartz | Retrieving device |
| CN107137890A (en) * | 2016-09-27 | 2017-09-08 | 五邑大学 | A kind of intelligent tennis ball pickup robot based on image recognition |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN108579030A (en) * | 2018-04-09 | 2018-09-28 | 宁波乔克兄弟三维科技有限公司 | A kind of robot of view-based access control model picks up ball method |
| CN108786041A (en) * | 2018-07-12 | 2018-11-13 | 贵州理工学院 | Intelligent ball collecting control method and intelligent ball collecting robot |
| CN109333549A (en) * | 2018-10-25 | 2019-02-15 | 湖南大学 | A kind of ping-pong pickup mobile robot and control method based on machine vision |
-
2019
- 2019-08-13 CN CN201910744905.2A patent/CN110433467B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160105647A1 (en) * | 2011-03-25 | 2016-04-14 | Eric D. Schwartz | Retrieving device |
| CN107137890A (en) * | 2016-09-27 | 2017-09-08 | 五邑大学 | A kind of intelligent tennis ball pickup robot based on image recognition |
| CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
| CN108579030A (en) * | 2018-04-09 | 2018-09-28 | 宁波乔克兄弟三维科技有限公司 | A kind of robot of view-based access control model picks up ball method |
| CN108786041A (en) * | 2018-07-12 | 2018-11-13 | 贵州理工学院 | Intelligent ball collecting control method and intelligent ball collecting robot |
| CN109333549A (en) * | 2018-10-25 | 2019-02-15 | 湖南大学 | A kind of ping-pong pickup mobile robot and control method based on machine vision |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111530041A (en) * | 2020-04-24 | 2020-08-14 | 贵州民族大学 | Robot ball picking method and system |
| CN111744161A (en) * | 2020-07-29 | 2020-10-09 | 哈尔滨理工大学 | Table tennis falling point detection and edge ball wiping judgment system |
| CN112587884A (en) * | 2020-12-24 | 2021-04-02 | 烟台南山学院 | Full-automatic ball picking robot ball picking method based on monocular vision |
| CN114879669A (en) * | 2022-04-28 | 2022-08-09 | 贵州民族大学 | Robot control method based on multi-target path planning and intelligent picking |
| CN114879669B (en) * | 2022-04-28 | 2024-05-28 | 贵州民族大学 | Robot control method based on multi-target path planning and intelligent pickup |
| CN115421522A (en) * | 2022-06-20 | 2022-12-02 | 南京信息工程大学 | Unmanned aerial vehicle coverage path planning method based on parallel self-adaptive ant colony algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110433467B (en) | 2021-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110433467A (en) | Picking up table tennis ball robot operation method and equipment based on binocular vision and ant group algorithm | |
| CN108401461A (en) | Three-dimensional mapping method, device and system, cloud platform, electronic equipment and computer program product | |
| CN110008851B (en) | Method and equipment for detecting lane line | |
| CN106826833A (en) | Independent navigation robot system based on 3D solid cognition technologies | |
| CN107491070A (en) | A kind of method for planning path for mobile robot and device | |
| CN103678754B (en) | Information processor and information processing method | |
| CN110243360A (en) | Method for constructing and positioning map of robot in motion area | |
| CN110084243B (en) | File identification and positioning method based on two-dimensional code and monocular camera | |
| CN106845515A (en) | Robot target identification and pose reconstructing method based on virtual sample deep learning | |
| CN108596974A (en) | Dynamic scene robot localization builds drawing system and method | |
| CN101619984A (en) | Mobile robot visual navigation method based on colorful road signs | |
| CN107362987A (en) | The robot method for sorting and system of a kind of view-based access control model | |
| CN111136669A (en) | Chess playing robot based on global vision and control method thereof | |
| CN113485350A (en) | Robot movement control method, device, equipment and storage medium | |
| CN111275015A (en) | Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system | |
| CN106295512A (en) | Many correction line indoor vision data base construction method based on mark and indoor orientation method | |
| CN109521781A (en) | Unmanned plane positioning system, unmanned plane and unmanned plane localization method | |
| CN102768767B (en) | Online three-dimensional reconstructing and locating method for rigid body | |
| CN111998862A (en) | Dense binocular SLAM method based on BNN | |
| EP3825804A1 (en) | Map construction method, apparatus, storage medium and electronic device | |
| CN115077563A (en) | Vehicle positioning accuracy evaluation method and device and electronic equipment | |
| CN106325278A (en) | Robot positioning and navigating method based on ellipse identification | |
| CN116171962B (en) | Efficient targeted spray regulation and control method and system for plant protection unmanned aerial vehicle | |
| CN108961393A (en) | A kind of human body modeling method and device based on point cloud data stream | |
| CN107845136A (en) | A kind of landform flattening method of three-dimensional scenic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20191112 Assignee: NANJING NANYOU INSTITUTE OF INFORMATION TECHNOVATION Co.,Ltd. Assignor: NANJING University OF POSTS AND TELECOMMUNICATIONS Contract record no.: X2021980014141 Denomination of invention: Operation method and equipment of table tennis picking robot based on binocular vision and ant colony algorithm Granted publication date: 20210212 License type: Common License Record date: 20211206 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: NANJING NANYOU INSTITUTE OF INFORMATION TECHNOVATION Co.,Ltd. Assignor: NANJING University OF POSTS AND TELECOMMUNICATIONS Contract record no.: X2021980014141 Date of cancellation: 20231107 |
|
| EC01 | Cancellation of recordation of patent licensing contract |