CN112149555A - Multi-storage AGV tracking method based on global vision - Google Patents
Multi-storage AGV tracking method based on global vision Download PDFInfo
- Publication number
- CN112149555A CN112149555A CN202010998015.7A CN202010998015A CN112149555A CN 112149555 A CN112149555 A CN 112149555A CN 202010998015 A CN202010998015 A CN 202010998015A CN 112149555 A CN112149555 A CN 112149555A
- Authority
- CN
- China
- Prior art keywords
- agv
- area
- agvs
- tracking
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- Quality & Reliability (AREA)
- General Business, Economics & Management (AREA)
- Development Economics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Game Theory and Decision Science (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-storage AGV tracking method based on global vision, which comprises the following steps: 1) shooting a global image of the warehouse and sending the global image to a control center; 2) processing the global image, identifying a plurality of AGVs by a tracking system in a first frame by using a target detection algorithm, and determining the ID and the pose of each AGV according to an Apriltag code at the top of each AGV; 3) dividing the warehouse into a plurality of areas, and after acquiring AGV pose information sent by a control center, a scheduling system performs path planning on each AGV by using a hierarchical planning algorithm; 4) combining the path information of each AGV with a multi-AGV tracking algorithm, predicting the position of the AGV, selecting the area where the AGV is located by using a boundary frame, and determining the information of each AGV; 5) and the control center transmits the speed instruction converted from the path information to the AGV, controls the AGV and completes the sorting task of the goods.
Description
Technical Field
The invention relates to the field of warehouse logistics for assisting in goods sorting, in particular to a multi-warehouse AGV tracking method based on global vision.
Background
At present, online shopping has become a trend, and along with electronic commerce flourishing together, the logistics industry also has been prosperous, but the logistics industry has been keen to advance, for example, the large throughput of goods in the logistics storage system makes the sorting and transportation work in the warehouse become a difficult problem, but the problems of low sorting efficiency and high error rate cannot be solved well by the investment of a large amount of labor cost, even the phenomenon of violent sorting can also occur, so that the heavy work in the current logistics storage can not be better dealt with by only manpower, and the automatic storage system is gradually prosperous. The intelligent warehousing system utilizes an intelligent navigation Vehicle (AGV) with strong flexibility to replace manual goods sorting and transportation, so that resources such as manpower are saved, sorting efficiency can be improved, and error rate is reduced, however, most of the existing intelligent navigation vehicles need to be provided with expensive sensors such as a camera or a laser radar to collect information and to be communicated with a control system independently, and warehouses also need to be correspondingly provided with devices such as magnetic nails and magnetic tapes, so that the problems of high cost, complex configuration, limited control efficiency and the like exist (Lynch L, Newe T, simulation J, et al. Some smart warehousing systems adopt global Vision monitoring (CN201911046869.9), and a method of tracking and scheduling by identifying AGV signatures, but still have problems such as high requirement of signatures for resolution or illumination, low fault tolerance, and less information storage, and the conventional detection and tracking method is inefficient (Dnmez E, Kocamaz AF, digital M.A Vision-Based Real-Time Mobile Robot Controller Design Based on Gaussian Function for index Environment [ J ]. array Journal for & Engineering,2017(4): 1-16), so the smart warehousing systems Based on global Vision also need to be improved in terms of identification and tracking.
Disclosure of Invention
The invention discloses a multi-storage AGV tracking method based on global vision, which is lower in configuration cost and higher in operation efficiency compared with a traditional AGV storage system. A camera is installed at the top end of a warehouse to shoot a plurality of AGVs for monitoring global images, the characteristic codes at the tops of the AGVs are used for detecting and identifying, the warehouse is divided into a plurality of areas and is scheduled according to transportation task requirements, a scheduling algorithm is integrated in a tracking process based on target detection, the system runs smoothly, the AGVs can be accurately tracked and scheduled in real time, and sorting transportation tasks are efficiently completed.
The invention is realized by at least one of the following technical schemes.
A multi-storage AGV tracking method based on global vision comprises the following steps:
1) a camera at the top of the storage system shoots a global image of the storage, and the global image is sent to a control center for tracking the storage AGV;
2) a tracking system of the control center processes the global image, a tiny-Yolov3 target detection network is used for identifying a plurality of AGVs in a first frame, and the ID and the pose of each AGV are determined according to an Apriltag code at the top of the AGVs;
3) dividing the warehouse into a plurality of areas, and after acquiring the AGV pose information sent by the tracking system, the scheduling system of the control center plans the path of each AGV by using a hierarchical planning algorithm;
4) after the dispatching system plans the path of each AGV, the tracking system directly predicts the position of the AGV by using the path information, then a dynamic window is set to frame the area where the AGV is located, the Apriltag code in the area is also identified to determine the information of each AGV, and if the number of the detected AGVs is less than the number of the AGVs in the current warehouse, the step 2 is executed again);
5) the control center sends the speed instruction converted from the path information to the AGV, and the AGV is controlled to reach the target area according to the established route so as to complete the sorting task of the goods.
Further, the top camera in step 1) is placed at the top of the warehouse and used for monitoring and shooting images of the whole warehouse, wherein a control center of the warehousing system comprises a multiple-AGV tracking system and a multiple-AGV scheduling system which are respectively used for determining pose information and specific paths of each AGV.
Further, the step 2) specifically comprises the following steps:
step one, when a warehousing system is started, because the position and path information of the AGVs is not available, a tiny _ YOLOv3 target detection network is used for identifying the AGVs, and tiny _ YOLOv3 target detection network is trained by AGV pictures, pedestrian pictures and common obstacle pictures in a warehouse;
step two, in order to identify the characteristic pattern at the top of the AGV, image processing is carried out on the AGV area selected by the frame, and the specific processing mode is that firstly, the area image is converted into a gray image, then, binary processing is carried out by utilizing Opencv, and then, a Canny edge detection algorithm and a FindContour function are used for extracting the outline of the object;
fitting the contour through an ApproxPolDP function, selecting a characteristic pattern with a proper area and determined as a square as a characteristic pattern of the current AGV, wherein the characteristic pattern selects an open-source AprilTag system;
rasterizing the whole square area based on four corner points of the outer contour of the April tag code at the top of the AGV, decoding the April tag according to pixel values of grids in the area, and further acquiring binary information stored by the current April tag code to be used as the ID of the current AGV;
after the tracing system extracts the outline set of the pattern, fitting each side of the square by using a least square method, obtaining a rotation angle through the slope of each side, and then weighting to obtain a proper angle:
wherein alpha isiThe deflection angle corresponding to the ith edge, n is a set of edge points (x) of the characteristic patternj,yj) The number of the pattern is alpha degrees, that is, any forward characteristic pattern is obtained, but the direction of the AGV is unknown, so that the pattern can be sequentially rotated by 90 degrees according to the rotational invariance of AprilTag, and finally the standard characteristic pattern in the family code is obtained, and then the angles of the two parts are added to obtain the heading angle of the AGV, as shown in the following formula:
θ＝90°×r+(45°-∝) (2)
wherein alpha is an auxiliary angle used for calculating the AGV heading angle, and the value is equal to the mean value of included angles of four edges of an AprilTag code; r is 0-3, which means that the current identification can be matched with the family code through r times of 90-degree rotation, kiThe weight of the angle is calculated for the ith edge least squares method, where each edge is set to have the same weight.
Further, step 3) comprises:
the warehouse is divided into two layers for classification and storage, wherein the second layer is a sorting layer, and the first layer is a centralized storage layer; dividing a sorting area into n areas, wherein each area comprises an intersection and a corridor connected with the intersection, the area also comprises a plurality of grids, the grids are the most basic units for AGV scheduling, and the grids are divided into a starting area and a target area on a sorting layer according to different division of labor of the areas; in the process of executing tasks, each AGV obtains an optimal path reaching a target area by utilizing layered planning according to allocated tasks, the AGV runs to the target area according to the path and then puts a carried package into a pipeline, goods can enter a first layer for centralized storage, and the AGV returns to an idle initial area after the sorting task is finished to perform next task circulation;
furthermore, the hierarchical planning means that when the AGV path planning is performed, the complexity of an algorithm is reduced, the scheduling efficiency is guaranteed, the complex multi-target planning problem is converted into two sub-problems, and the two sub-problems are calculated by using two algorithms respectively; the first layer is planning among areas, the area where the AGV is located is set as an initial area, the area where the package needs to be delivered is set as a target area, and a shortest area path set which connects the initial area and the target area is calculated by utilizing an A-path planning algorithm; and the second planning is the planning among grids, and the grid resources in the same region are distributed by using a time window arrangement algorithm, so that the AGV can rapidly leave the current region according to the region path set on the basis of no collision.
Further, the step 4) is specifically as follows:
on the premise that the position and path information of all AGVs are known, a tiny-yolov3 target detection network is not used for detection, the position of the current AGV is directly predicted according to the path planned by a scheduling system, a dynamic window is set on the basis of the predicted position to frame an area where the AGV may exist, the operation of the step 2) is also carried out on the area, Apriltag codes at the top of the AGVs are used for positioning, after all the AGVs are detected, if the number of the identified AGVs is not consistent with the initial number, the AGV does not run according to a set track, the tiny-yolov3 target detection network is reused for identifying the AGV in the global image, any derailed AGV can be repositioned as long as the AGV is still in the current storage environment, and is scheduled in the control center to return to the set track again.
Further, the step 5) is as follows:
in order to enable large-scale AGVs in a warehouse to stably operate, the AGVs are set to run at constant speed, so that the linear speed sent to each AGV by the control center is a fixed value, in addition, the control center changes the angular speed which is sent to the AGV in real time according to the AGV course angle and the coordinates which are calculated by the multiple AGV tracking systems, and corrects the angular speed on the basis of the linear speed in a mode that if the current position of the AGV is far away from a set route, the linear speed value is increased, and if the current position of the AGV exceeds the set route, the linear speed is reduced, so that the AGV is controlled not to deviate from the set.
The invention processes the global image by using opencv, detects the target by using tiny _ YOLOv3, uses Apriltag feature code for AGV identification, ensures the identification rate and fault tolerance, combines the target detection algorithm with the path planning algorithm, tracks based on prediction and improves the overall planning efficiency.
Compared with the prior art, the invention has the following advantages:
(1) by utilizing a machine vision technology, the central controller acquires all AGV information by processing a global image shot by the top camera, so that the calculation amount of the AGV vehicle-mounted controller is reduced, the information interaction amount is reduced, and the overall efficiency of the system is improved;
(2) the neural network is combined with the traditional image processing technology to track the multiple AGVs, the neural network is utilized to ensure the positioning precision of the tracking algorithm, the processing speed of the algorithm is accelerated through the traditional image technology, and finally the purpose of accurately tracking the multiple AGVs in real time is achieved.
(3) The AprilTag system is used for identifying and positioning the AGV, so that higher accuracy is achieved;
(4) the fuzzy prediction position is obtained by combining the AGV historical position information and the planned path, the image of the whole warehouse does not need to be processed, meanwhile, whether the AGV derails or not can be monitored, unnecessary information processing is reduced, and the tracking efficiency is improved;
(5) if the AGV breaks away from the track, the control system can detect the AGV again in the global range, so that the AGV breaking away from the set route returns as soon as possible, and manual operation is reduced.
Drawings
FIG. 1 is a schematic diagram of a warehousing system based on global vision according to the present embodiment;
FIG. 2 is a schematic diagram of a Gazebo simulation of the warehousing system based on global vision in this embodiment;
FIG. 3 is a structural diagram of the warehousing system based on global vision according to the embodiment;
FIG. 4 is a flow chart of the multiple AGV tracking system of the present embodiment;
FIG. 5 is a diagram of a topology diagram of a simulation warehouse and a topology diagram of a region based on global vision according to the embodiment;
FIG. 6 is an AGV with an Apriltag code on top according to this embodiment;
FIG. 7 is a diagram of the AprilTag code of the present embodiment;
FIG. 8 is a tracking result of the multiple AGV tracking system according to this embodiment;
FIG. 9 is a graph comparing deviation of AGV heading angle calculated by different tracking methods.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited to these examples.
FIG. 1 is a global vision based multiple AGV storage system comprising: image acquisition equipment (industrial camera), WiFi, data storage server, computer (control center), storage AGV.
The image acquisition equipment is arranged at the top of the warehouse and used for acquiring global images of the warehouse and transmitting the acquired global warehouse images to the data storage server for storage through a network. The computer is connected with the data storage server through the Internet, and can directly read or delete the task information of the image and the package stored in the server; the computer (control center) includes many AGV tracking system and many AGV dispatch systems, control center can handle and the analysis to the image information who reads, tracks the position appearance information of AGV in the warehouse system, in sending all AGV's information to dispatch systems simultaneously, output every AGV's control information, control command passes through in WIFI transmits corresponding AGV, AGV moves according to control command, in delivering the express delivery parcel to the pipeline, be used for concentrated storage, and then reach the purpose of letter sorting parcel.
Fig. 2 is a global vision based warehouse system built in Gazebo simulation software. The warehouse is divided into two layers for classification and storage. The AGV transports the package on the correct conveyor and to the target area. After completing the classified task, the AGV may return to the start area to ship additional packages and begin a new task again. In fig. 1, there are four cameras at the top of the warehouse, each camera taking a respective area. Then, the warehouse is equally divided into four parts, and images shot by a plurality of cameras are spliced to obtain a global image of the warehouse.
As shown in fig. 3, a multi-storage AGV tracking method based on global vision includes the following steps:
1) shooting a global image of the warehouse by using a camera at the top of the warehousing system, and sending the global image to a control center for tracking the warehousing AGV;
2) using a tiny _ YOLOv3 target detection network or a scheduling system to frame out the area where each AGV is located, and sequentially identifying AprilTag codes in the areas to obtain coordinates, a heading angle and a number of the AGVs, as shown in fig. 4, the specific process is as follows:
step one, dividing a simulated warehouse system into a plurality of areas for AGV management and scheduling; the specific process is as follows:
as shown in FIG. 5, the current simulation warehouse is divided into 16 areas, respectively (Q)1,Q2…,Q16). Each area has an intersection and a corridor connected to it. As shown in the right diagram indicated by arrows in fig. 5, each of the areas (area 6, area 7, area 10, area 11) including the intersection is composed of 4 corridors (Cor1, Cor2, Cor3, Cor4) and an intersection, and their basic units are grids, and in order that a plurality of AGVs do not collide during the scheduling process, the grids are arranged to have an area slightly larger than that of the AGVs. White part is the free area that can pass through in the storage map, and regional 4, regional 8, regional 14 and regional 16 are AGV task initiation area, and black region is the pipeline of transportation goods, and AGV needs to transport goods to corresponding region, and then accomplish the letter sorting task in throwing into the pipeline to the goods.
Step two, processing the global image, identifying a plurality of AGVs by the tracking system at a first frame by using a tiny _ YOLOv3 target detection network, and determining the ID and the pose of each AGV according to an Apriltag code at the top of the AGVs, as shown in FIG. 6; the specific process is as follows:
when the warehousing system starts, since the position information and the path information of the AGVs are not available, the tiny _ YOLOv3 target detection network is used for detecting the AGVs. The Tiny _ YOLOv3 target detection network was trained with AGV pictures, pedestrian pictures, and pictures of common obstacles in the warehouse. Through the tiny _ YOLOv3 target detection network, all AGVs can be identified in the warehousing global image of the first frame, and meanwhile, the system can also detect warehouse workers and other obstacles and send the information of the AGVs to the control center.
In order to identify the characteristic pattern at the top of the AGV, the system carries out image processing on the AGV area selected by the frame, firstly, the area image is converted into a gray image, then, the Opencv is utilized to carry out binarization processing, and then, a Canny edge detection algorithm and a FindContour function are used for extracting the outline of the object;
and fitting the contour through an ApproxPolyDP function, and selecting the characteristic pattern with a proper area and judged to be a square as the characteristic pattern of the current AGV. In the AprilTag system with the characteristic pattern selection open source, AprilTag codes have advantages in recognition rate and fault tolerance compared with other characteristic codes, and different types of AprilTag codes, such as 16h5,25h9,36h11 and the like, can be selected according to the required storage information amount and the requirements on the characteristic code recognition rate and the fault tolerance. The first number is the number of bits of the AprilTag code that can store information, and the second number is the minimum Hamming distance between any two codes in the family code of the type. Taking 36h11 as an example, the number of bits of each codeword in the family code is 36, and the minimum Hamming distance between any two codewords is 11. The family code can correct 5 bit errors and detect 5 bit errors. Meanwhile, the method also has rotation specificity, namely when one feature pattern is rotated by a plurality of 90 degrees, the Hamming distance between a decoding result and the family code is still larger than a set value, only the feature code in the unique direction can be matched with the decoding result of the family code, and the characteristic can be used for determining the rotation angle of the feature code. In addition, different from the two-dimensional code, the positioning is carried out through a peripheral black rectangular frame, and compared with three positioning patterns of the two-dimensional code, the robustness is better under the condition of low resolution.
The entire square area is then divided into 64 grids based on the four corner points of the aprilat code outline at the top of the AGV, i.e., points a, B, C, and D in fig. 7. And decoding the April tag according to the pixel value of each grid in the area (the black grid indicates that the bit is 1, and the white grid indicates that the bit is 0), and further acquiring binary information stored by the current April tag code as the ID of the current AGV.
Since the corner points of the feature pattern are easily blurred, the determination of the pattern rotation angle by using the corner points is different from the conventional AprilTag, and after the tracing system extracts the outline set of the pattern, each side of the square is fitted by using a least square method. The angle of rotation is found by the slope of each side and then weighted to obtain the appropriate angle.
Wherein alpha isiThe deflection angle corresponding to the ith side is shown in fig. 7. n is a set of edge points (x) of the characteristic patternj,yj) The number of the cells. Any one forward characteristic pattern can be obtained by rotating the current pattern by alpha degrees, but at the moment, the direction of the AGV is unknown, so that the standard characteristic pattern which is consistent with the standard characteristic pattern in the family code can be obtained by sequentially rotating the pattern by 90 degrees according to the rotation specificity of AprilTag. The angle of these two portions is then added to obtain the AGV heading angle, as shown in the following equation:
θ＝90°×r+(45°-∝) (2)
where alpha is an assist angle for calculating the AGV heading angle,the value is equal to the mean of the included angles of the four edges of the AprilTag code. r belongs to 0-3, and represents that the current identification can be matched with the family code through r times of 90-degree rotation. k is a radical ofiThe weight of the angle is calculated for each edge least squares method, where each edge is set to have the same weight.
The effect of the multiple AGV tracking system is shown in fig. 8, and 20 AGVs in the storage system can be selected and then output their course angles and numbers in real time for scheduling of multiple AGVs.
Step three, after determining the serial number and the attitude information of each AGV, performing layered planning on each AGV to obtain the path information of the AGV; the specific process is as follows:
the hierarchical planning means that when the AGV path planning is carried out, the complexity of an algorithm is reduced, meanwhile, the scheduling efficiency is guaranteed, the complex multi-target planning problem is converted into two sub-problems, and the two algorithms are used for calculation respectively. The first layer is planning among areas, wherein the area where the AGV is located is set as a starting area, the area where the package needs to be delivered is set as a target area, and a shortest area path set which connects the starting area and the target area is calculated by using an A-path planning algorithm. And the second planning is the planning among grids, and the grid resources in the same region are distributed by using a time window arrangement algorithm, so that the AGV can rapidly leave the current region according to the region path set on the basis of no collision.
Step four, predicting a boundary frame of the current frame AGV by using the path information, then directly identifying an ApriTag code in the boundary frame, when the detected number of the AGVs is insufficient, alarming by a system, recording the number of the missing AGV, and detecting the AGV by using a tiny _ YOLOv3 target detection network again; the specific process is as follows:
on the premise of knowing all AGV positions and path information, a tiny _ Yolov3 target detection network is not used for detection, the position of the current AGV is directly predicted according to the path planned by the scheduling system, and a boundary frame is arranged on the basis of the position to frame out the possible areas of the AGV, so that the real-time performance of the tracking algorithm is greatly improved.
The bounding box is then similarly subjected to step two, using the AprilTag code at the top of the AGV to locate. And when all the boundary frames are detected, if the number of the identified AGVs is consistent with the actual number of the AGVs, circulating the process to track and dispatch the multiple AGVs in real time. If the identified AGV number is not consistent with the initial number, it means that there is an AGV that does not run according to the predetermined trajectory, for example, the AGV is out of order, or the AGV is hijacked. At this time, the tiny _ YOLOv3 target detection network is reused to identify the AGVs in the global image, and any derailed AGVs can be repositioned as long as the AGVs are still in the current storage environment, and are scheduled in the control center to return to the established tracks again.
In order to improve the robustness of the system, besides the AprilTag code has fault tolerance, the system adopts a hierarchical programming algorithm, and the information of each AGV is registered in each area, so that if the hamming distance of the identified AprilTag code is greater than a set threshold value, the code has the possibility of error identification, and then the hamming distance between the code and other AprilTag codes at the top of the AGV in the area is calculated, and the result with the minimum hamming distance is selected as the ID of the current AGV, so that the probability of error identification of the AGV is greatly reduced.
For example, when the AGV itself is in zone 1, which registers 4 AGVs, numbered 1(d5d628584), 2(d97f18b49), 3(dd280910e), and 4(e479e9c98), respectively. In the region, the minimum Hamming distance of 4 code words is 14, the minimum Hamming distance of other bits is 16 and 17, and the error detection and correction capability is improved from 5, 5 to 6, 6. And if a 7-bit error occurs. To misidentify as 3, the error must be at the intersection of 1 and 3, which is a very low probability, thus greatly increasing the recognition rate of the AGV.
3) The control center generates instructions to each AGV according to the AGV information obtained by the tracking system and the scheduling system in a targeted manner, and controls all the AGVs to move according to the specified path; the specific process is as follows:
and controlling the plurality of AGVs by using the ROS robot operating system, and carrying out distributed communication based on the ROS system through WIFI to send a control instruction to each AGV in real time. In order to ensure that large-scale AGVs in the warehouse stably operate, the AGVs are arranged to run at a constant speed. Therefore, the linear velocity sent to each AGV by the control center is a fixed value, and besides, the angular velocity to be sent to the AGV is changed in real time by the control center according to the AGV heading angle and the coordinates calculated by the multiple AGV tracking systems, and is corrected on the basis of the linear velocity. If the current position of the AGV is far away from the set route, the linear speed value is increased, and if the current position of the AGV exceeds the set route, the linear speed value is decreased. In this manner, the AGV is controlled not to deviate from the designated route.
This example has the following advantages:
1. compared with the method using the triangle built-in numbers as the feature patterns, the aprilat code adopted by the embodiment has stronger fault tolerance, and can be correctly identified even if a part of the pattern is missing. Meanwhile, the AprilTag code has stronger information storage capacity and can mark 587 or even more AGVs at the same time.
2. Compared with the method for positioning multiple AGVs by directly using an image processing technology, the method for positioning the AGVs by using the tiny-YOLOv3 has higher real-time performance and can adapt to different light sources and storage environments. Meanwhile, common obstacles in the warehouse, such as warehouse managers or dropped goods, can be identified in real time through tiny-YOLOv3, and collision of the AGVs is avoided.
3. The output of a conventional multi-target tracking scheme is the type and coordinates of the target. The multiple AGV tracking algorithm provided by the embodiment has the function of outputting the target rotation angle in real time, which is necessary for AGV navigation.
4. Compared with the method for calculating the AGV heading angle by using the two-dimensional codes and the triangles as the characteristic patterns, the algorithm provided by the embodiment has higher calculation accuracy. To explain this by experiment, a two-dimensional code of version 1(21 × 21 module) that is most easily recognized and a triangle with a built-in number are placed on the top of the AGV as a feature pattern. The two-dimensional code determines the rotation angle through three positioning codes, and the triangular pattern calculates the rotation angle through extracting three corner points of the triangular pattern. The same speed instruction is sent to the AGV, the top camera shoots a global image in the AGV moving process, and then deviation values of the AGV course angle and the real course angle calculated by different algorithms in continuous 100 frames of images are compared, and the result is shown in fig. 9. By comparison, the multi-AGV tracking method provided by the invention has higher accuracy in calculating the course angle of the AGV.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (7)
1. A multi-storage AGV tracking method based on global vision is characterized by comprising the following steps:
1) a camera at the top of the storage system shoots a global image of the storage, and the global image is sent to a control center for tracking the storage AGV;
2) a tracking system of the control center processes the global image, a tiny-Yolov3 target detection network is used for identifying a plurality of AGVs in a first frame, and the ID and the pose of each AGV are determined according to an Apriltag code at the top of the AGVs;
3) dividing the warehouse into a plurality of areas, and after acquiring the AGV pose information sent by the tracking system, the scheduling system of the control center plans the path of each AGV by using a hierarchical planning algorithm;
4) after the dispatching system plans the path of each AGV, the tracking system directly predicts the position of the AGV by using the path information, then a dynamic window is set to frame the area where the AGV is located, the Apriltag code in the area is also identified to determine the information of each AGV, and if the number of the detected AGVs is less than the number of the AGVs in the current warehouse, the step 2 is executed again);
5) the control center sends the speed instruction converted from the path information to the AGV, and the AGV is controlled to reach the target area according to the established route so as to complete the sorting task of the goods.
2. The method for tracking the AGVs in multiple warehouses based on the global vision as claimed in claim 1, wherein a top camera is placed on the top of the warehouse in step 1) and used for monitoring and shooting images of the whole warehouse, and a control center of the warehouse system comprises a multiple AGV tracking system and a multiple AGV scheduling system which are respectively used for determining the pose information and the specific path of each AGV.
3. The multi-storage AGV tracking method based on global vision as claimed in claim 1, wherein the step 2) comprises the following steps:
step one, when a warehousing system is started, because the position and path information of the AGVs is not available, a tiny _ YOLOv3 target detection network is used for identifying the AGVs, and tiny _ YOLOv3 target detection network is trained by AGV pictures, pedestrian pictures and common obstacle pictures in a warehouse;
step two, in order to identify the characteristic pattern at the top of the AGV, image processing is carried out on the AGV area selected by the frame, and the specific processing mode is that firstly, the area image is converted into a gray image, then, binary processing is carried out by utilizing Opencv, and then, a Canny edge detection algorithm and a FindContour function are used for extracting the outline of the object;
fitting the contour through an ApproxPolDP function, selecting a characteristic pattern with a proper area and determined as a square as a characteristic pattern of the current AGV, wherein the characteristic pattern selects an open-source AprilTag system;
rasterizing the whole square area based on four corner points of the outer contour of the April tag code at the top of the AGV, decoding the April tag according to pixel values of grids in the area, and further acquiring binary information stored by the current April tag code to be used as the ID of the current AGV;
after the tracing system extracts the outline set of the pattern, fitting each side of the square by using a least square method, obtaining a rotation angle through the slope of each side, and then weighting to obtain a proper angle:
wherein alpha isiThe deflection angle corresponding to the ith side, n is a characteristic patternOne edge point set (x)j,yj) The number of the pattern is alpha degrees, that is, any forward characteristic pattern is obtained, but the direction of the AGV is unknown, so that the pattern can be sequentially rotated by 90 degrees according to the rotational invariance of AprilTag, and finally the standard characteristic pattern in the family code is obtained, and then the angles of the two parts are added to obtain the heading angle of the AGV, as shown in the following formula:
θ＝90°×r+(45°-∝) (2)
wherein alpha is an auxiliary angle used for calculating the AGV heading angle, and the value is equal to the mean value of included angles of four edges of an AprilTag code; r is 0-3, which means that the current identification can be matched with the family code through r times of 90-degree rotation, kiThe weight of the angle is calculated for the ith edge least squares method, where each edge is set to have the same weight.
4. The method of claim 1, wherein step 3) comprises:
the warehouse is divided into two layers for classification and storage, wherein the second layer is a sorting layer, and the first layer is a centralized storage layer; dividing a sorting area into n areas, wherein each area comprises an intersection and a corridor connected with the intersection, the area also comprises a plurality of grids, the grids are the most basic units for AGV scheduling, and the grids are divided into a starting area and a target area on a sorting layer according to different division of labor of the areas; in the process of executing the tasks, each AGV can obtain an optimal path reaching a target area according to the distributed tasks by utilizing layered planning, the AGV runs to the target area according to the path and then puts a package to be carried into a pipeline, goods can enter a first layer for centralized storage, and the AGV returns to an idle starting area again after the sorting task is finished to perform next task circulation.
5. The method for tracking the AGV according to the claim 4, wherein the hierarchical planning means that when the AGV path planning is performed, the complexity of the algorithm is reduced, the scheduling efficiency is guaranteed, the complex multi-target planning problem is converted into two sub-problems, and the two sub-problems are calculated by using the two algorithms respectively; the first layer is planning among areas, the area where the AGV is located is set as an initial area, the area where the package needs to be delivered is set as a target area, and a shortest area path set which connects the initial area and the target area is calculated by utilizing an A-path planning algorithm; and the second planning is the planning among grids, and the grid resources in the same region are distributed by using a time window arrangement algorithm, so that the AGV can rapidly leave the current region according to the region path set on the basis of no collision.
6. The method for tracking the AGV according to claim 1, wherein the step 4) comprises the following steps:
on the premise that the position and path information of all AGVs are known, a tiny-yolov3 target detection network is not used for detection, the position of the current AGV is directly predicted according to the path planned by a scheduling system, a dynamic window is set on the basis of the predicted position to frame an area where the AGV may exist, the operation of the step 2) is also carried out on the area, Apriltag codes at the top of the AGVs are used for positioning, after all the AGVs are detected, if the number of the identified AGVs is not consistent with the initial number, the AGV does not run according to a set track, the tiny-yolov3 target detection network is reused for identifying the AGV in the global image, any derailed AGV can be repositioned as long as the AGV is still in the current storage environment, and is scheduled in the control center to return to the set track again.
7. The method for tracking the AGV according to claim 1, wherein the step 5) comprises the following steps:
in order to enable large-scale AGVs in a warehouse to stably operate, the AGVs are set to run at constant speed, so that the linear speed sent to each AGV by the control center is a fixed value, in addition, the control center changes the angular speed which is sent to the AGV in real time according to the AGV course angle and the coordinates which are calculated by the multiple AGV tracking systems, and corrects the angular speed on the basis of the linear speed in a mode that if the current position of the AGV is far away from a set route, the linear speed value is increased, and if the current position of the AGV exceeds the set route, the linear speed is reduced, so that the AGV is controlled not to deviate from the set.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010873730 | 2020-08-26 | ||
| CN2020108737308 | 2020-08-26 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112149555A true CN112149555A (en) | 2020-12-29 |
| CN112149555B CN112149555B (en) | 2023-06-20 |
Family
ID=73893657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010998015.7A Active CN112149555B (en) | 2020-08-26 | 2020-09-21 | Global vision-based multi-warehouse AGV tracking method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112149555B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113218403A (en) * | 2021-05-14 | 2021-08-06 | 哈尔滨工程大学 | AGV system of inertia vision combination formula location |
| CN113343962A (en) * | 2021-08-09 | 2021-09-03 | 山东华力机电有限公司 | Visual perception-based multi-AGV trolley working area maximization implementation method |
| CN113378701A (en) * | 2021-06-08 | 2021-09-10 | 华南理工大学 | Ground multi-AGV state monitoring method based on unmanned aerial vehicle |
| CN113516429A (en) * | 2021-04-08 | 2021-10-19 | 华南理工大学 | Multi-AGV global planning method based on network congestion model |
| CN113689056A (en) * | 2021-10-23 | 2021-11-23 | 深圳市骏丰木链网科技股份有限公司 | Intelligent warehouse management method and system based on supply chain |
| CN114253229A (en) * | 2021-11-25 | 2022-03-29 | 云南昆船智能装备有限公司 | AGV-based flexible conveying system and conveying method |
| CN114489062A (en) * | 2022-01-18 | 2022-05-13 | 北京理工大学 | Workshop logistics-oriented multi-automatic guided vehicle distributed dynamic path planning method |
| WO2022237399A1 (en) * | 2021-05-13 | 2022-11-17 | 北京京东乾石科技有限公司 | Control method and apparatus for robot, and robot |
| CN115646818A (en) * | 2022-12-28 | 2023-01-31 | 江苏智联天地科技有限公司 | AGV intelligence letter sorting system |
| CN115860642A (en) * | 2023-02-02 | 2023-03-28 | 上海仙工智能科技有限公司 | Access management method and system based on visual identification |
| CN116451888A (en) * | 2022-10-21 | 2023-07-18 | 中国科学院沈阳自动化研究所 | Multi-AVG-based flexible production workshop collaborative scheduling method |
| CN117093009A (en) * | 2023-10-19 | 2023-11-21 | 湖南睿图智能科技有限公司 | Logistics AGV trolley navigation control method and system based on machine vision |
| CN117877276A (en) * | 2024-03-11 | 2024-04-12 | 天津联汇智造科技有限公司 | Method for controlling logistics traffic in factory |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108469786A (en) * | 2018-01-26 | 2018-08-31 | 西安电子科技大学 | Extensive intelligent storage distribution radio frequency |
| CN108534788A (en) * | 2018-03-07 | 2018-09-14 | 广州大学 | A kind of AGV air navigation aids based on kinect visions |
| CN110780671A (en) * | 2019-10-30 | 2020-02-11 | 华南理工大学 | Storage navigation intelligent vehicle scheduling method based on global vision |
| CN111274934A (en) * | 2020-01-19 | 2020-06-12 | 上海智勘科技有限公司 | Implementation method and system for intelligently monitoring forklift operation track in warehousing management |
-
2020
- 2020-09-21 CN CN202010998015.7A patent/CN112149555B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108469786A (en) * | 2018-01-26 | 2018-08-31 | 西安电子科技大学 | Extensive intelligent storage distribution radio frequency |
| CN108534788A (en) * | 2018-03-07 | 2018-09-14 | 广州大学 | A kind of AGV air navigation aids based on kinect visions |
| CN110780671A (en) * | 2019-10-30 | 2020-02-11 | 华南理工大学 | Storage navigation intelligent vehicle scheduling method based on global vision |
| CN111274934A (en) * | 2020-01-19 | 2020-06-12 | 上海智勘科技有限公司 | Implementation method and system for intelligently monitoring forklift operation track in warehousing management |
Non-Patent Citations (3)
| Title |
|---|
| WEI XIE 等: "An Improved Heuristic Path Planning Algorithm for Minimizing Energy Consumption in Distributed Multi-AGV Systems", 《IEEE》, pages 1 - 6 * |
| YINDONG LIAN 等: "Improved A* Multi-AGV Path Planning Algorithm Based on Grid-Shaped Network", 《IEEE》, pages 2088 - 2092 * |
| YINDONG LIAN: ":A Probabilistic Time-Constrained Based Heuristic Path Planning Algorithm in Warehouse Multi-AGV systems", 《IFAC》, pages 2538 - 2542 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113516429A (en) * | 2021-04-08 | 2021-10-19 | 华南理工大学 | Multi-AGV global planning method based on network congestion model |
| CN113516429B (en) * | 2021-04-08 | 2024-03-29 | 华南理工大学 | Multi-AGV global planning method based on network congestion model |
| WO2022237399A1 (en) * | 2021-05-13 | 2022-11-17 | 北京京东乾石科技有限公司 | Control method and apparatus for robot, and robot |
| CN113218403A (en) * | 2021-05-14 | 2021-08-06 | 哈尔滨工程大学 | AGV system of inertia vision combination formula location |
| CN113218403B (en) * | 2021-05-14 | 2022-09-09 | 哈尔滨工程大学 | AGV system of inertia vision combination formula location |
| CN113378701B (en) * | 2021-06-08 | 2023-02-14 | 华南理工大学 | Ground multi-AGV state monitoring method based on unmanned aerial vehicle |
| CN113378701A (en) * | 2021-06-08 | 2021-09-10 | 华南理工大学 | Ground multi-AGV state monitoring method based on unmanned aerial vehicle |
| CN113343962A (en) * | 2021-08-09 | 2021-09-03 | 山东华力机电有限公司 | Visual perception-based multi-AGV trolley working area maximization implementation method |
| CN113689056A (en) * | 2021-10-23 | 2021-11-23 | 深圳市骏丰木链网科技股份有限公司 | Intelligent warehouse management method and system based on supply chain |
| CN114253229A (en) * | 2021-11-25 | 2022-03-29 | 云南昆船智能装备有限公司 | AGV-based flexible conveying system and conveying method |
| CN114253229B (en) * | 2021-11-25 | 2023-12-05 | 云南昆船智能装备有限公司 | AGV-based flexible conveying system and conveying method |
| CN114489062B (en) * | 2022-01-18 | 2023-03-21 | 北京理工大学 | Workshop logistics-oriented multi-automatic guided vehicle distributed dynamic path planning method |
| CN114489062A (en) * | 2022-01-18 | 2022-05-13 | 北京理工大学 | Workshop logistics-oriented multi-automatic guided vehicle distributed dynamic path planning method |
| CN116451888A (en) * | 2022-10-21 | 2023-07-18 | 中国科学院沈阳自动化研究所 | Multi-AVG-based flexible production workshop collaborative scheduling method |
| CN116451888B (en) * | 2022-10-21 | 2024-01-09 | 中国科学院沈阳自动化研究所 | Flexible production workshop cooperative scheduling method based on multiple AGVs |
| CN115646818A (en) * | 2022-12-28 | 2023-01-31 | 江苏智联天地科技有限公司 | AGV intelligence letter sorting system |
| CN115860642A (en) * | 2023-02-02 | 2023-03-28 | 上海仙工智能科技有限公司 | Access management method and system based on visual identification |
| CN117093009A (en) * | 2023-10-19 | 2023-11-21 | 湖南睿图智能科技有限公司 | Logistics AGV trolley navigation control method and system based on machine vision |
| CN117093009B (en) * | 2023-10-19 | 2024-02-09 | 湖南睿图智能科技有限公司 | Logistics AGV trolley navigation control method and system based on machine vision |
| CN117877276A (en) * | 2024-03-11 | 2024-04-12 | 天津联汇智造科技有限公司 | Method for controlling logistics traffic in factory |
| CN117877276B (en) * | 2024-03-11 | 2024-05-24 | 天津联汇智造科技有限公司 | Method for controlling logistics traffic in factory |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112149555B (en) | 2023-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112149555B (en) | Global vision-based multi-warehouse AGV tracking method | |
| CN110780671B (en) | Storage navigation intelligent vehicle scheduling method based on global vision | |
| US11176388B2 (en) | Tracking vehicles in a warehouse environment | |
| CN106950972B (en) | Automatic Guided Vehicle (AGV) and route correction method thereof | |
| JP7531053B2 (en) | Warehouse system, shuttle vehicle for warehouse system, and navigation method thereof | |
| US20200133283A1 (en) | Proceedable direction detection apparatus and proceedable direction detection method | |
| CN111198496A (en) | Target following robot and following method | |
| CN117496739B (en) | Self-adaptive double-layer garage navigation system for transfer robot | |
| Behrje et al. | An autonomous forklift with 3d time-of-flight camera-based localization and navigation | |
| CN117531717A (en) | Patrol type intelligent garbage sorting robot and working method thereof | |
| CN118092457B (en) | Storage AGV commodity circulation dolly operation track control system | |
| CN114012732B (en) | Robot control method, device, equipment and medium | |
| CN113485318B (en) | Position correction method, device, equipment and storage medium | |
| US12055938B2 (en) | Mobile robot, mobile robot control system, and mobile robot control method | |
| CN115877853B (en) | Intelligent warehouse logistics path planning system and method | |
| CN114675658B (en) | Automatic empty shelf recycling method, system, equipment and medium | |
| CN113673276A (en) | Target object identification docking method and device, electronic equipment and storage medium | |
| US20220162001A1 (en) | Predicting a path of material handling equipment and determining an obstacle-free path | |
| CN114564024A (en) | Forklift AGV laser obstacle avoidance control method | |
| Bhat et al. | An advanced autonomous forklift based on a networked control system | |
| CN113741412A (en) | Control method and device for automatic driving equipment and storage medium | |
| CN113093766B (en) | Position deviation correcting method, device and equipment for logistics transportation equipment and storage medium | |
| CN118168573B (en) | Automatic-driving unmanned delivery vehicle navigation and path planning method and system | |
| Ho et al. | Pose-Based Visual Servoing with Lightweight Deep-Learning Binarization for Autonomous Mobile Robot Application | |
| CN117877276B (en) | Method for controlling logistics traffic in factory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |