WO2022132025A1 - Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur - Google Patents

Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur Download PDF

Info

Publication number
WO2022132025A1
WO2022132025A1 PCT/SG2020/050743 SG2020050743W WO2022132025A1 WO 2022132025 A1 WO2022132025 A1 WO 2022132025A1 SG 2020050743 W SG2020050743 W SG 2020050743W WO 2022132025 A1 WO2022132025 A1 WO 2022132025A1
Authority
WO
WIPO (PCT)
Prior art keywords
pallet
identifier
inventory
images
stream
Prior art date
Application number
PCT/SG2020/050743
Other languages
English (en)
Inventor
Chin Kok Yap
Wen Da YEO
Govind SUNDAR
Li Sheng Ryan YAP
Original Assignee
Ych Group Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ych Group Pte Ltd filed Critical Ych Group Pte Ltd
Priority to PCT/SG2020/050743 priority Critical patent/WO2022132025A1/fr
Publication of WO2022132025A1 publication Critical patent/WO2022132025A1/fr

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography

Definitions

  • the present disclosure relates to inventory systems for indoor warehouses. More particularly, the present disclosure relates to autonomous drone inventory of palleted collections placed within pallet bays of an indoor warehouse.
  • Indoor warehouses organize multiple inventory items within palleted collections for storage.
  • the contents of each palleted collection is recorded into a warehouse database, with pallet content records listing the inventory identifiers of the individual inventory items assembled into each palleted collection.
  • the palleted collection are typically given a physical pallet label representing a pallet identifier.
  • the pallet identifier is printed as a pallet label in a machine readable format such as a QR code or a linear barcode.
  • Storage locations within the warehouse are identified using location labels representing location identifiers (also in a machine readable format such as a QR code or a linear barcode).
  • warehouse inventory systems are used by warehouse staff to memorialize the storage locations of the palleted items.
  • a barcode reader in data communication with the warehouse database is used by warehouse staff to update the warehouse database by scanning in both a current location identifier of the new storage location and the pallet identifier of the palleted collection.
  • supplemental inventory checks are periodically performed. These supplemental inventory checks are often performed by warehouse staff via a time-intensive manual inspection of the palleted collections within the indoor warehouse. Excessive height of the multi-tiered racks, narrow aisles, and ongoing activity of forklifts within the indoor warehouse can make these supplemental inventory checks both time-consuming and potentially dangerous.
  • Supplemental inventory processes performed by electric drones in the prior art are ineffective due to their excessive need for human guidance and slow progress (stopping and hovering at each pallet bay) through the indoor warehouse.
  • Electric drones have a limited flight time, often in the range of only ten to twenty minutes. Stopping and hovering at each pallet bay and relying on skilled human piloting of the electric drones limits the effectiveness of the prior art.
  • the prior art poses difficulties in implementation of supplemental inventories, given the 24/7 activity within indoor warehouses and the ever increasing size of multi-tiered indoor warehouses. What is needed is a system and method for performing an autonomous drone inventory using a preset programmable flight path and a barcode reading capability (and database integration protocol) that does not require discontinuous movement of the electric drone in its path through the indoor warehouse.
  • an electric drone can conduct an unfettered fly through an indoor warehouse along the tiers of multi-tiered racks to collect pallet identifier information in a short timeframe that does not significantly interfere with the ongoing operations of the indoor warehouse.
  • the invention is a system or method for autonomous drone inventory of palleted collections of inventory items placed within pallet bays of multitiered racks of an indoor warehouse.
  • a navigation module is configured to store a programmable flight path passing alongside a pre-selection of pallet bays, navigate flight of an electric drone along the programmable flight path, during the flight along the programmable flight path capture a stream of images of pallet labels placed on the palleted collections, and associate each image with a location determined from the location markers.
  • a recognition processor is configured to recognize a pallet identifier on each of the pallet labels captured within the stream of images and link the pallet identifier to one of the pallet bays.
  • a reporting module outputs an inventory location report identifying the pallet bay storing each inventory item.
  • a first embodiment of the invention is an autonomous drone inventory system for an indoor warehouse with a plurality of location markers, the indoor warehouse storing a plurality of palleted collections of inventory items on multi-tiered racks, each palleted collection including a pallet label displaying a pallet identifier, each palleted collection located in a pallet bay associated with a bay identifier, each pallet bay locatable by at least one of a plurality of location markers mounted in the indoor warehouse, the system comprising: (a) an electric drone including a propulsion system, a directional camera, and a data output; (b) a navigation module; (c) a computing device with a data input, a user interface, and a recognition processor; (d) a server with a warehouse database; and (e) a reporting module.
  • the navigation module is configured to: (i) store a three- dimensional map of the multi-tiered racks and a programmable flight path configured to pass alongside a pre-selection of pallet bays on the multi-tiered racks; (ii) navigate the electric drone along the programmable flight path; (iii) aim the directional camera toward the pallet label of each palleted collection located along the programmable flight path, wherein the directional camera captures a stream of images during the programmable flight path; and (iv) associate each of the images in the stream of images with an image location and an image timestamp.
  • the data input is configured for data communication with the data output of the electric drone to receive the stream of images captured by the directional camera of the electric drone along the programmable flight path.
  • the recognition processor is configured to offload or execute a recognition process, the recognition process including the steps of: (1) recognizing each pallet identifier from the pallet labels captured within the stream of images; (2) identifying the bay identifier of each pallet identifier captured within the stream of images based upon the image location associated with at least one of the images employed to recognize the pallet identifier; and (3) creating and storing a pallet location record within a flight inventory database for each pallet identifier recognized within the stream of images.
  • Each pallet location record memorializes, for each pallet identifier identified by the recognition processor from the stream of images during a programmable flight path: (1) the pallet identifier identified by the recognition processor; (2) the bay identifier for the pallet bay associated with the pallet identifier; and (3) the image timestamp of at least one of the images employed to recognize the pallet identifier.
  • the warehouse database includes: (1) a plurality of inventory records, each inventory record including a linked inventory identifier and an inventory description; and (2) a plurality of pallet content records, each pallet content record including a linked pallet identifier and a list of inventory identifiers associated with the linked pallet identifier.
  • the server is configured to: (1) download or link to each pallet location record of the flight inventory database; (2) link each pallet location record with one of the pallet content records via a first cross reference; (3) link each pallet content record with at least one of the inventory records via a second cross reference.
  • the first cross reference is between: (a) the pallet identifier in the pallet location record of the flight inventory database; and (b) the linked pallet identifier in one of the pallet content records of the warehouse database.
  • the second cross reference is between: (a) each of the inventory identifiers listed in each pallet content record of the warehouse database; and (b) the linked inventory identifier of each of the inventory records of the warehouse database.
  • the reporting module is configured to: (i) determine the bay identifier associated with each inventory identifier via the first cross reference; (ii) determine the inventory description associated with each linked inventory identifier via the second cross reference; and (iii) output an inventory location report.
  • the inventory location report lists each inventory identifier linked with: (1) the bay identifier associated with the inventory identifier as determined by the first cross reference; and (2) the inventory description associated with the linked inventory identifier as determined by the second cross reference.
  • a second embodiment of the invention is computer-implemented method for conducting an autonomous drone inventory of an indoor warehouse with a plurality of location markers, the indoor warehouse storing a plurality of palleted collections of inventory items on multi-tiered racks, each palleted collection including a pallet label displaying a pallet identifier, each palleted collection located in a pallet bay associated with a bay identifier, each pallet bay locatable by at least one of a plurality of location markers mounted in the indoor warehouse.
  • the method comprising the steps of: (a) maintaining an electric drone including a propulsion system, a directional camera, and a data output; (b) configuring a navigation module; (c) maintaining a computing device with a data input, a user interface, and a recognition processor; (d) maintaining a warehouse database; (e) downloading or linking to each pallet location record of the flight inventory database; (f) linking each pallet location record with one of the pallet content records via a first cross reference; (g) linking each pallet content record with at least one of the inventory records via a second cross reference; (h) determining the bay identifier associated with each inventory identifier via the first cross reference; (i) determining the inventory description associated with each linked inventory identifier via the second cross reference; and (j) outputting an inventory location report.
  • the navigation module is configure to: (i) store a three-dimensional map of the multi-tiered racks and a programmable flight path configured to pass alongside a pre-selection of pallet bays on the multi-tiered racks; (ii) navigate the electric drone along the programmable flight path; (iii) aim the directional camera toward the pallet label of each palleted collection located along the programmable flight path, wherein the directional camera captures a stream of images during the programmable flight path; and (iv) associate each of the images in the stream of images with an image location and an image timestamp.
  • the data input is configured for data communication with the data output of the electric drone to receive the stream of images captured by the directional camera of the electric drone along the programmable flight path.
  • the recognition processor is configured to offload or execute a recognition process, the recognition process including the steps of (1) recognizing each pallet identifier from the pallet labels captured within the stream of images; (2) identifying the bay identifier of each pallet identifier captured within the stream of images based upon the image location associated with at least one of the images employed to recognize the pallet identifier; and (3) creating and storing a pallet location record within a flight inventory database for each pallet identifier recognized within the stream of images.
  • Each pallet location record memorializes, for each pallet identifier identified by the recognition processor from the stream of images during a programmable flight path: (1) the pallet identifier identified by the recognition processor; (2) the bay identifier for the pallet bay associated with the pallet identifier; and (3) the image timestamp of at least one of the images employed to recognize the pallet identifier.
  • the warehouse database includes: (i) a plurality of inventory records, each inventory record including a linked inventory identifier and an inventory description; and (ii) a plurality of pallet content records, each pallet content record including a linked pallet identifier and a list of inventory identifiers associated with the linked pallet identifier.
  • the first cross reference is between: (i) the pallet identifier in the pallet location record of the flight inventory database; and (ii) the linked pallet identifier in one of the pallet content records of the warehouse database.
  • the second cross reference is between: (i) each of the inventory identifiers listed in each pallet content record of the warehouse database; and (ii) the linked inventory identifier of each of the inventory records of the warehouse database.
  • the inventory location report lists each inventory identifier linked with: (i) the bay identifier associated with the inventory identifier as determined by the first cross reference; and (ii) the inventory description associated with the linked inventory identifier as determined by the second cross reference.
  • Technical objects of the invention include: (a) pre-programming the flight path of the electric drone through the indoor warehouse using location markers mounted in the indoor warehouse for navigation of the electric drone; (b) collecting a stream of images of pallet labels and location information during a continuous fly through along the tiers of the multi-tiered racks, unfettered by stop and hover requirements; (c) performing a recognition processing step upon a stream of images taken by the electric drone without impact to the flight path or flight time of the electric drone; and (d) cross referencing flight inventory data records to warehouse database records to obtain the location of inventory items within the pallet bays of the multi-tiered racks.
  • FIG. 1 is a block diagram representing an autonomous drone inventory system in an embodiment of the invention.
  • FIG. 2 is an illustration of the collection and processing of a stream of images of pallet labels in an embodiment of the invention.
  • FIG. 3 is an illustration of a programmable flight path navigated by an electric drone within a multi-tiered racks of an indoor warehouse using location markers in an embodiment of the invention.
  • FIG. 4 is a flowchart of the steps taken in an embodiment of the invention.
  • the software and hardware of a “computing device” may be implemented within a dedicated electric drone control unit, a smart phone, a tablet, a laptop, a single stand-alone computer, or a stand-alone server.
  • a “user interface” may be implemented by a display monitor, a keyboard, a mouse, a touch screen, a touch pad, and/or similarly directed means.
  • the user interface may be configured by a dedicated electric drone control unit, a smart phone, a tablet, a laptop, a single stand-alone computer, or a stand-alone server.
  • the software and hardware of a “server” may be implemented within a single stand-alone computer, a stand-alone server, multiple dedicated servers, and/or a virtual server running on a larger network of servers and/or a cloud-based service.
  • a “database” may store data to and access data from a single stand-alone computer, a laptop, a tablet, a data server, multiple dedicated data servers, a cloud-based service, and/or a virtual server running on a network of servers.
  • database indicate a collection of tables, records, and/or linkage information for the data records.
  • each database can be maintained separately and/or maintained collectively in a single database or through linkages to other database.
  • the databases depicted in the description and figures can be on the same server or on separate servers. Data links between tables can be links in one database or links between separate databases.
  • the data output 12 of an electric drone 10 is in data communication with a data input 21 of a computing device 20 through a router 41. As depicted in FIG. 1, the data output 12 and the data input 21 are both wireless transceivers.
  • the computing device 20 is also in data communication with a server 50 via the router 41 and a network 40.
  • the server 50 includes a warehouse database 51.
  • the warehouse database 51 includes pallet content records 52 and inventory records 53.
  • the computing device 20 includes a user interface 22, a recognition processor 23, and a flight inventory database.
  • the electric drone 10 includes a directional camera 13 for capturing an image of a pallet label 31 placed on a palleted collection 30.
  • the palleted collection 30 is located in a pallet bay 32.
  • the invention can also use wired data exchange, such as with the electric drone 10 at its dedicated staging area during battery charging.
  • FIG. 2 is an illustration of the collection and processing of stream of images 33 of pallet labels 31 in an embodiment of the invention.
  • FIG. 2 depicts an electric drone 10 with a directional camera 13 capturing an image of a pallet label 31 placed on a palleted collection 30.
  • the data output 12 of the electric drone 10 is configured to transfer a stream of images 33 of pallet labels 31 to a recognition processor 23.
  • Information derived from the recognition processor 23 is organized in pallet location records of a flight inventory database 24.
  • Each pallet location record includes a pallet identifier 25A, a bay identifier 25B, and an image timestamp 25C.
  • a warehouse database includes a pallet identifier 25A, a bay identifier 25B, and an image timestamp 25C.
  • the pallet content records 52 includes pallet content records 52 and inventory records 53.
  • the pallet content records 52 include pallet content records 52 and inventory records 53.
  • the inventory records 53 each include a linked inventory identifier 53A and an inventory description 53B.
  • each pallet location record 25 is linked to an associated pallet content record 52 via a first cross reference of: (i) the pallet identifier 25A of the pallet location record; and (ii) the linked pallet identifier 52A of the pallet content records 52.
  • These two record fields are meant to both identify the same pallet collection.
  • the pallet identifiers 25A and the linked pallet identifier 52A fields can be the same number or the same alphanumeric identifier. Alternatively, these fields can be linkable via an algorithm, a hash, an encryption process, a masked version of one field, or an additional record crossreferencing table for the two fields (hereafter “linkable equivalent”).
  • each pallet content record 52 is linked to an associated inventory record 53 via a second cross reference of (i) each of the inventory identifiers 52B listed in each pallet content record 52; and (ii) the linked inventory identifier 53A of the inventory records 53.
  • These two record fields are meant to both identify the same inventory items.
  • the listed inventory identifiers 52B and linked inventory identifiers 53A fields can use the same number or the same alphanumeric identifier. Alternatively, these fields can be linkable via a linkable equivalent (as defined above).
  • the data for the bay identifier 25B for a specific palleted collection 30 is taken from the results of the autonomous drone inventory.
  • the data for the current location identifier 52C is created by a warehouse inventory system (such as via manual barcoding by warehouse staff) during operations within the indoor warehouse. These two identifiers can be identical or linkable equivalents.
  • the bay identifier 25B of the pallet location record 25 created from the results of an autonomous drone inventory for a specific palleted collection 30 should be the same (or linkable equivalent) of the current location identifier 52C of the pallet content records 52 for that specific palleted collection 30.
  • the warehouse staff should update the location of a palleted collection 30 in the warehouse database 51 using the warehouse inventory system each time the palleted collection 30 is moved. If the palleted collection 30 is correctly placed in a pallet bay 32 and the warehouse database 51 records are accurate, then the autonomous drone inventory results should confirm such. If there has been an incorrect placement of a palleted collection 30 or an incorrect warehouse database 51 record entry, then this error should likewise be evident from the autonomous drone inventory results.
  • the autonomous drone inventory system 1-00 is useful as a supplemental inventory (or audit) of the warehouse database 51 records. As errors are found, the autonomous drone inventory results can be utilized to update warehouse operation procedures, faulty equipment, warehouse staff supervision, and/or warehouse staff training to minimize future errors.
  • Electric drone 10 flight times must be minimized. Many jurisdictions required human observation of all indoor flights of electric drone 10 and the removal of human personnel from the areas under the flight path. Also the flight time of electric drones 10 is often only ten to twenty minutes in length (with battery recharge times often hours in length required there between). Smooth, orderly, predictable, and short time length flights of the electric drones 10 is therefore a critical feature of the invention.
  • the invention allows the recognition processing and assembly of flight inventory database 24 data records to be performed asynchronously from the capture of images of the pallet labels 31 by the electric drones 10. Errors between the bay identifier 25B and the current location identifier 52C for a palleted collection 30, for instance, will not slow or alter the programmable flight path (62A to 62E).
  • the database record-keeping tasks performed by the invention are intentionally separated from the electric drone’s 10 task of collecting the stream of images 33. This separation allows the programmable flight path (62A to 62E) and electric drone 10 components to be optimized for a rapid collection of quality images of the pallet labels 31. The supplemental inventory performed by the electric drones 10 is therefore not disruptive to the operations of the indoor warehouse.
  • a programmable flight path (62A to 62E), for instance, may be conducted once per day or several times per day without significant impact upon the operations of the indoor warehouse. Between each supplemental inventory, the system 1-00 is given more than sufficient time to perform recognition processing, database record updates, and reporting.
  • warehouse staff can address data inconsistencies flagged by the inventory location report.
  • the warehouse staff may be directed to manually inspect palleted collections 30 flagged in the inventory location report to research the causes of the errors and correct those errors.
  • FIG. 3 is an illustration of a programmable flight path (62A to 62E) navigated by an electric drone 10 within a multi-tiered racks 3-00 of an indoor warehouse using location markers 61 in an embodiment of the invention.
  • the multi-tiered rack 3-00 depicted in FIG. 3 includes three columns (columns A, B, and C) from left to right.
  • the multi-tiered rack 3-00 depicted in FIG. 3 includes four tiers (tiers 1, 2, 3, and 4) from bottom to top.
  • Location markers 61 are mounted on the multi-tiered racks 3-00 at locations notated by: (i) Al, A2, A3, and A4 in the first column; (ii) Bl, B2, B3, and B4 in the second column; and (iii) Cl, C2, and C3 in the third column.
  • a programmable flight path (62A, 62B, 62C, 62D, and 62E) of the electric drone 10 is depicted as following right to left in along the topmost tier 4, then left to right along tier 3, then right to left along tier 2, and then left to right along bottom tier 1.
  • the programmable flight path (62A to 62E) is intended to pass along each pallet label 31 placed on the palleted collections 30 located in the pallet bays 32.
  • pallet bay 32 is a term used to indicate the location where a palleted collection 30 is placed on the multi-tiered racks 3-00. E.g., each pallet bay 32 is not necessarily afforded its own enclosed area in a column. Similar to the depiction in FIG. 3, a shelf area on a tier of one column may include two palleted collections 30 side by side. With variance in the sizing of inventory items and pallet configurations, the pallet bay 32 dimensions may vary within an indoor warehouse. In its most general form, the pallet bay 32 term is used to indicate a location where a palleted collection 30 can reside.
  • the location markers 61 in FIG. 3 are depicted as near field communication beacons.
  • the location markers 61 can also be visual location labels (identified via the directional camera 13 and/or a navigational camera) or warehouse fixtures (identified via the directional camera 13, a navigational camera, and/or a LIDAR sensor).
  • the location markers 61 can also be launch point markers on the floor of each aisle, each launch point marker indicating a launch point for the electric drone 10 in the aisle.
  • RFID tags are not suitable generally as location markers 61 in an indoor warehouse.
  • the RFID reader protocols often include asynchronous reading of RFID tags located in an area. Also the metal in the multi-tiered racks 3-00 tends to interfere with the RF signals used for RFID tag identification.
  • the location markers 61 in FIG. 3 are organized by tiers and columns on the multi-tiered racks 3-00.
  • the location markers 61 can also be mounted on other fixtures of the indoor warehouse and/or the floor of the indoor warehouse.
  • the location markers 61 can also be mounted in grid organizations other than by aisle, tier, or column. Combinations of different types of locations can be used in unison also to identify a pallet bay 32.
  • the navigation module can also leverage the inertial navigation system of the electric drone 10 to assist with the determination of the electric drone’ s 10 x, y, and z axis location within the indoor warehouse.
  • the inertial navigation system can include multi-axis accelerometers, magnetometers, and/or speed measuring devices.
  • the electric drone 10, for instance, can be place at a launch point indicated by a launch point marker facing a predefined direction then directed though all, or a portion, of the programmable flight path (62A to 62E).
  • Inertial navigation may be augmented by a laser scanner, LIDAR and/or height sensors mounted on the electric drone 10.
  • a location marker can be as simple as a launch point marker for a launch point of the electric drone 10, where the launch point marker can be, for instance, an “X” painted on the floor at one end of each aisle.
  • the electric drone 10 can be pointed in an initial direction at launch.
  • GPS is not suitable for an indoor warehouse given the overhead roofing and multi-tiered racks 3-00 blocking or distorting the GPS satellite signals.
  • the programmable flight path (62A to 62E) can be updated based on the observed effectiveness and speed of previous flights.
  • An effective programmable flight path (62A to 62E) will balance the benefits of a short flight time with the need to adequately gather quality images of the pallet labels 31 during flight. Thus in-field optimizations made by warehouse staff to the programmable flight path (62A to 62E) are likely to be productive.
  • Possible adjustments to the programmable flight path (62A to 62E) could include: (i) reducing or increasing the distance between the directional camera 13 and the pallet labels 31 during flights; (ii) matching of the speed of the electric drone 10 across the tiers with the imaging capabilities of the directional camera 13; (iii) switching between use of real time data downloads during the flight of the stream of images 33 and alternatively employing post-flight data downloads of the stream of images 33 at the electric drone’s 10 dedicated staging area during battery charging; and/or (iv) reducing or increasing the length of the electric drone’s 10 flight to account for deterioration of battery storage or propulsion system 11 efficacy of the electric drone 10.
  • FIG. 4 is a flowchart of the steps taken in an embodiment of the invention. Steps 4-01 to 4-09 are listed below. 4-01 install location markers 61 within the indoor warehouse with sufficient density to permit identification of all pallet bays 32 using sensor information collected along the programmable flight path (62A to 62E) by location sensors mounted on the electric drone 10
  • each pallet label 31 along the programmable flight path (62A to 62E) can be adequately captured by a directional camera 13 mounted on the electric drone 10 within the flight time of a single battery charge of the electric drone 10 4-04 pre-schedule the programmable flight path (62A to 62E) during a time the path of the programmable flight path (62A to 62E) is clear
  • 4-06 image process the stream of images 33 to identify each palleted collection 30 by its pallet label 31 captured in the images and link each pallet identifier 25A with its location in a pallet bay 32
  • 4-08 run an inventory location report from: (i) a first cross reference 54 of the pallet identifier 25A of the location records 25 in the flight inventory database 24 with the linked pallet identifier 52A of the pallet content records 52 of the warehouse database 51; and (ii) a second cross reference 55 to the inventory records 53 of a warehouse database 51
  • Step 4-09 the results of the autonomous drone inventory are useful to identifying any errors in the pallet content records 52 or inventory records 53.
  • One possible error is an absence of a specific palleted collection 30 at a specific pallet bay 32.
  • the current location identifier 52C field of the associated pallet content record 52 indicates that the specific palleted collection 30 should be located at a specific pallet bay 32 but the autonomous drone inventory results do not locate it at that pallet bay 32 location.
  • a second possible error would be the presence of a specific palleted collection
  • the invention is a system 1-00 or method for autonomous drone inventory of palleted collections 30 of inventory items placed within pallet bays 32 of multi-tiered racks 3-00 of an indoor warehouse.
  • a navigation module is configured to store a programmable flight path (62A to 62E) passing alongside a pre-selection of pallet bays 32, navigate flight of an electric drone 10 along the programmable flight path (62A to 62E), during the flight along the programmable flight path (62A to 62E) capture a stream of images 33 of pallet labels 31 placed on the palleted collections 30, and associate each image with a location determined from the location markers 61.
  • a recognition processor 23 is configured to recognize a pallet identifier 25A on each of the pallet labels
  • a reporting module outputs an inventory location report identifying the pallet bay 32 storing each inventory item.
  • a first embodiment of the invention is an autonomous drone inventory system 1-00 for an indoor warehouse with a plurality of location markers 61, the indoor warehouse storing a plurality of palleted collections 30 of inventory items on multi-tiered racks 3- 00, each palleted collection 30 including a pallet label 31 displaying a pallet identifier 25A, each palleted collection 30 located in a pallet bay 32 associated with a bay identifier 25B, each pallet bay 32 locatable by at least one of a plurality of location markers 61 mounted in the indoor warehouse, the system 1-00 comprising: (a) an electric drone 10 including a propulsion system 11, a directional camera 13, and a data output 12; (b) a navigation module; (c) a computing device 20 with a data input 21, a user interface 22, and a recognition processor 23; (d) a server 50 with a warehouse database 51; and (e) a reporting module.
  • the navigation module is configured to: (i) store a three-dimensional map of the multi-tiered racks 3-00 and a programmable flight path (62A to 62E) configured to pass alongside a pre-selection of pallet bays 32 on the multi-tiered racks 3- 00; (ii) navigate the electric drone 10 along the programmable flight path (62A to 62E); (iii) aim the directional camera 13 toward the pallet label 31 of each palleted collection 30 located along the programmable flight path (62A to 62E), wherein the directional camera 13 captures a stream of images 33 during the programmable flight path (62A to 62E); and (iv) associate each of the images in the stream of images 33 with an image location and an image timestamp 25C.
  • the data input 21 is configured for data communication with the data output 12 of the electric drone 10 to receive the stream of images 33 captured by the directional camera 13 of the electric drone 10 along the programmable flight path (62A to 62E).
  • the recognition processor 23 is configured to offload or execute a recognition process, the recognition process including the steps of: (1) recognizing each pallet identifier 25A from the pallet labels 31 captured within the stream of images 33; (2) identifying the bay identifier 25B of each pallet identifier 25A captured within the stream of images 33 based upon the image location associated with at least one of the images employed to recognize the pallet identifier 25A; and (3) creating and storing a pallet location record within a flight inventory database 24 for each pallet identifier 25A recognized within the stream of images 33.
  • Each pallet location record memorializes, for each pallet identifier 25A identified by the recognition processor 23 from the stream of images 33 during a programmable flight path (62A to 62E): (1) the pallet identifier 25A identified by the recognition processor 23; (2) the bay identifier 25B for the pallet bay 32 associated with the pallet identifier 25A; and (3) the image timestamp 25C of at least one of the images employed to recognize the pallet identifier 25A.
  • the warehouse database 51 includes: (1) a plurality of inventory records 53, each inventory record 53 including a linked inventory identifier 53A and an inventory description 53B; and (2) a plurality of pallet content records 52, each pallet content record 52 including a linked pallet identifier 52A and a list of inventory identifiers 52B associated with the linked pallet identifier 52A.
  • the server 50 is configured to: (1) download or link to each pallet location record of the flight inventory database 24; (2) link each pallet location record with one of the pallet content records 52 via a first cross reference; (3) link each pallet content record 52 with at least one of the inventory records 53 via a second cross reference.
  • the first cross reference is between: (a) the pallet identifier 25A in the pallet location record of the flight inventory database 24; and (b) the linked pallet identifier 52A in one of the pallet content records 52 of the warehouse database 51.
  • the second cross reference is between: (a) each of the inventory identifiers 52B listed in each pallet content record 52 of the warehouse database 51; and (b) the linked inventory identifier 53A of each of the inventory records 53 of the warehouse database 51.
  • the reporting module is configured to: (i) determine the bay identifier 25B associated with each inventory identifier 52B via the first cross reference; (ii) determine the inventory description 53B associated with each linked inventory identifier 53A via the second cross reference; and (iii) output an inventory location report.
  • the inventory location report lists each inventory identifier 52B linked with: (1) the bay identifier 25B associated with the inventory identifier 52B as determined by the first cross reference; and (2) the inventory description 53B associated with the linked inventory identifier 53A as determined by the second cross reference.
  • a second embodiment of the invention is computer-implemented method for conducting an autonomous drone inventory of an indoor warehouse with a plurality of location markers 61, the indoor warehouse storing a plurality of palleted collections 30 of inventory items on multi-tiered racks 3-00, each palleted collection 30 including a pallet label 31 displaying a pallet identifier 25A, each palleted collection 30 located in a pallet bay 32 associated with a bay identifier 25B, each pallet bay 32 locatable by at least one of a plurality of location markers 61 mounted in the indoor warehouse.
  • the method comprising the steps of: (a) maintaining an electric drone 10 including a propulsion system 11, a directional camera 13, and a data output 12; (b) configuring a navigation module; (c) maintaining a computing device 20 with a data input 21, a user interface 22, and a recognition processor 23; (d) maintaining a warehouse database 51; (e) downloading or linking to each pallet location record of the flight inventory database 24; (f) linking each pallet location record with one of the pallet content records 52 via a first cross reference; (g) linking each pallet content record 52 with at least one of the inventory records 53 via a second cross reference; (h) determining the bay identifier 25B associated with each inventory identifier 52B via the first cross reference; (i) determining the inventory description 53B associated with each linked inventory identifier 53A via the second cross reference; and (j) outputting an inventory location report.
  • the navigation module is configure to: (i) store a three-dimensional map of the multi-tiered racks 3-00 and a programmable flight path (62A to 62E) configured to pass alongside a pre-selection of pallet bays 32 on the multi-tiered racks 3-00; (ii) navigate the electric drone 10 along the programmable flight path (62A to 62E); (iii) aim the directional camera 13 toward the pallet label 31 of each palleted collection 30 located along the programmable flight path (62A to 62E), wherein the directional camera 13 captures a stream of images 33 during the programmable flight path (62A to 62E); and (iv) associate each of the images in the stream of images 33 with an image location and an image timestamp 25C.
  • the data input 21 is configured for data communication with the data output 12 of the electric drone 10 to receive the stream of images 33 captured by the directional camera 13 of the electric drone 10 along the programmable flight path (62A to 62E).
  • the recognition processor 23 is configured to offload or execute a recognition process, the recognition process including the steps of (1) recognizing each pallet identifier 25A from the pallet labels 31 captured within the stream of images 33; (2) identifying the bay identifier 25B of each pallet identifier 25A captured within the stream of images 33 based upon the image location associated with at least one of the images employed to recognize the pallet identifier 25A; and (3) creating and storing a pallet location record within a flight inventory database 24 for each pallet identifier 25A recognized within the stream of images 33.
  • Each pallet location record memorializes, for each pallet identifier 25A identified by the recognition processor 23 from the stream of images 33 during a programmable flight path (62A to 62E): (1) the pallet identifier 25A identified by the recognition processor 23; (2) the bay identifier 25B for the pallet bay 32 associated with the pallet identifier 25A; and (3) the image timestamp 25C of at least one of the images employed to recognize the pallet identifier 25A.
  • the warehouse database 51 includes: (i) a plurality of inventory records 53, each inventory record 53 including a linked inventory identifier 53A and an inventory description 53B; and (ii) a plurality of pallet content records 52, each pallet content record 52 including a linked pallet identifier 52A and a list of inventory identifiers 52B associated with the linked pallet identifier 52A.
  • the first cross reference is between: (i) the pallet identifier 25A in the pallet location record of the flight inventory database 24; and (ii) the linked pallet identifier 52A in one of the pallet content records 52 of the warehouse database 51.
  • the second cross reference is between: (i) each of the inventory identifiers 52B listed in each pallet content record 52 of the warehouse database 51; and (ii) the linked inventory identifier 53A of each of the inventory records 53 of the warehouse database 51.
  • the inventory location report lists each inventory identifier 52B linked with: (i) the bay identifier 25B associated with the inventory identifier 52B as determined by the first cross reference; and (ii) the inventory description 53B associated with the linked inventory identifier 53A as determined by the second cross reference.
  • each pallet content record 52 includes a current location identifier 52C inputted into the warehouse database 51 through a warehouse inventory system; and (b) the reporting module is further configured to check for errors and flag errors in the inventory location report.
  • the check for errors is between: (1) the bay identifier 25B of each palleted collection 30 in the pallet location records 25 of the flight inventory database 24; and (2) the current location identifier 52C of each palleted collection 30 in the pallet content records 52 of the warehouse database 51.
  • each pallet label 31 includes: (i) a QR code design identifying a first pallet identifier, the QR code design having side dimensions in the range of 10 to 30 centimeters; (ii) a linear barcode design identifying a second pallet identifier; and (iii) an RFID tag identifying a third pallet identifier.
  • the associations between the first pallet identifier, the second pallet identifier, and the third pallet identifier are stored in at least one of: (i) the flight inventory database 24; and (ii) the warehouse database 51.
  • the large sizing of the QR code (10 to 30 centimeters) minimizes the practical limitations of an indoor warehouse environment. For instance, palleted collections 30 may reside in the indoor warehouse for months or years, thereby suffering distorting from the collecting of dirt, dust, and/or moisture. Also wrapping materials used to hold the inventory items together or the pallet label 31 in place may cause visual distortion. With a larger QR code, for instance, wrinkles or blotches in the QR code are less likely to cause mis-reads or failed-reads for the QR code. These alternative identifying means are helpful in assisting the logistical demands of an indoor warehouse. The invention, for instance, is likely to rely on equipment and software from multiple vendors.
  • the electric drone 10 and computing device 20 are likely to be purchased from a separate vendor from the providers of the warehouse database 51.
  • Warehouse workers may be using RFID or linear barcode readers in most of their interactions with the palleted items. It is possible that the large QR codes on the pallet labels 31 are only used by the autonomous drone inventory system 1-00.
  • each location marker 61 mounted in the indoor warehouse identifies at least one of a launch point, an aisle, a tier, a column, and a pallet bay 32.
  • This type of placement of the location markers 61 creates a robust foundation for the design of the three-dimensional map and the programmable flight path (62A to 62E). It also enables more precision in the location of each pallet bay 32.
  • a predetermined location or a launch point marker can be used to indicate the launch point; a launch point marker can be, for instance, a “X” marked on the floor at one end of each aisle.
  • location markers 61 mounted in the indoor warehouse include at least one of: (a) a launch point marker; (b) a plurality of near field communication beacons identifiable by a near field communication sensor mounted on the electric drone 10; (c) a plurality of visual location labels; and (d) a plurality of warehouse fixtures identified on the three-dimensional map of the multi-tiered racks 3-00.
  • the visual location labels include at least one of a QR label, and an ArUco label, and a linear barcode label.
  • the navigation module is configured to identify each visual location label in at least one of: (1) the stream of images 33 from the directional camera 13; and (2) an alternative stream of images 33 captured by a navigational camera mounted on the electric drone 10.
  • the navigation module is configured to identify each warehouse fixture in at least one of: (i) the stream of images 33 from the directional camera 13; (ii) the alternative stream of images 33 from the navigational camera; and (iii) a LIDAR sensor data stream from a LIDAR sensor mounted on the electric drone 10.
  • each aisle can be cordoned off and its pallet bays 32 scanned separately in a series of separate aisle flights.
  • This approach can simplify the designs of the navigation module, the three-dimensional map, and the programmable flight path (62A to 62E).
  • the three-dimensional map is a compilation of separate aisle maps
  • the programmable flight path (62A to 62E) is a compilation of separate aisle flights.
  • This approach may be more effective for conducting an inventory during working hours of the indoor warehouse (which for many warehouses is 24 / 7), as the inconvenience to warehouse staff is minimized. Also, safety is enhanced because operation of the electric drone 10 during each separate aisle flight within a single aisle can be easily observed by designated warehouse staff for each separate aisle flight of the programmable flight path (62A to 62E).
  • the stream of images 33 captured by the directional camera 13 is downloaded in real time from the electric drone 10 to the computing device 20 via a data pathway during navigation of the electric drone 10 through the programmable flight path (62A to 62E).
  • the data pathway passes: (i) wirelessly from the data output 12 of the electric drone 10 to a router 41 in the indoor warehouse; and (ii) from the router 41 to the data input 21 of the computing device 20.
  • the computing device 20 is configured to display the stream of images 33 on the user interface 22 of the computing device 20 in real time.
  • an alternate stream of images is captured by a navigational camera mounted on the electric drone 10 and downloaded in real time from the electric drone 10 to the computing device 20 via a data pathway during navigation of the electric drone 10 through the programmable flight path (62A to 62E).
  • the data pathway passes: (i) wirelessly from the data output 12 of the electric drone 10 to a router 41 in the indoor warehouse; and (ii) from the router 41 to the data input 21 of the computing device 20.
  • the computing device 20 is configured to display the alternate stream of images on the user interface 22 of the computing device 20 in real time.
  • This embodiment has the advantage of using the stream of images 33 from an alternate stream of images taken by a navigation camera.
  • the alternate stream of images thus can be dedicated to an optimal viewing angle of the flight (likely parallel to the direction of flight of the electric drone 10).
  • warehouse safety and/or electric drone 10 flight safety rules may require this manner of dedicated real time forward viewing angle during programmable flight path (62A to 62E) operation.
  • each programmable flight path (62A to 62E) commences and finishes at a dedicated staging area with a charging dock; and (b) each navigation of the electric drone 10 through the programmable flight path (62A to 62E) to capture the stream of images 33 is prescheduled by the navigation module.
  • This embodiment is advantageous for efficient charging of the electric drone 10 between flights.
  • the battery storage time of an electric drone 10 may require multiple flights interspersed between battery recharging cycles to perform an inventory of the entire indoor warehouse.
  • the dedicated staging area can also be placed in a secure location protecting the electric drone 10 from theft, vandalism, and equipment movement within the indoor warehouse.
  • software and hardware of the navigation module is stored or mounted within at least one of: (a) the electric drone 10; (b) the computing device 20; and (c) a server 50.
  • Specifics of the navigation equipment and protocols for electric drones 10 differs between vendors. Some vendors include provide full autonomous navigation capability within the electronics and firmware of the electric drone 10. Other vendors split the navigation processing responsibility between the electric drone 10 and a computing device 20 (such as a laptop running the vendor’s navigation software). The invention is meant to be compatible with this type of flexibility in the design of the electric drones 10.
  • the electric drones 10 are primarily tasked with flying the programmed flight path and capturing the stream of images 33 of the pallet labels 31.
  • the recognition processor 23 of the computing device 20 is configured to direct the upload of the stream of images 33 to a server 50; (b) the recognition processor 23 offloads the recognition process to the server 50; and (c) the flight inventory database 24 is located on the server 50.
  • the invention allows the recognition processing of the stream of images 33 to be performed on the computing device 20 or offloaded to a server 50.
  • the advantage of performing the recognition processing at the computing device 20 is that the results of an autonomous drone inventory can be performed locally within the indoor warehouse without uploading the stream of images 33 to a server 50 through the network 40.
  • the disadvantage, however, is that the computing device 20 (which could be a laptop or a smart phone) may not have sufficient processing power, stored battery power, or data storage to perform the recognition processing in a reasonable amount of time.
  • performance of the recognition processing at a server 50 may be a better option for many implementations of the invention, especially for larger indoor warehouses or indoor warehouses with a high volume of individual palleted collections 30.
  • an audit copy of the stream of images 33 can be archived at the server 50 for each autonomous drone inventory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Economics (AREA)
  • Multimedia (AREA)
  • Operations Research (AREA)
  • Accounting & Taxation (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Finance (AREA)
  • Quality & Reliability (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Development Economics (AREA)
  • Automation & Control Theory (AREA)

Abstract

L'invention concerne un système (1-00) ou un procédé pour un inventaire par drone autonome d'ensembles sur palettes (30) d'articles d'inventaire placés à l'intérieur de baies pour palettes (32) de râteliers à plusieurs étages (3-00) d'un entrepôt en intérieur. Un module de navigation est configuré pour stocker une trajectoire de vol programmable (62A à 62E) passant à côté d'une pré-sélection de baies pour palettes (32), naviguer en vol un drone électrique (10) le long de la trajectoire de vol programmable (62A à 62E), pendant le vol le long de la trajectoire de vol programmable (62A à 62E) capturer un flux d'images (33) d'étiquettes de palette (31) placées sur les ensemble sur palettes (30), et associer chaque image à un emplacement déterminé à partir des marqueurs d'emplacement (61). Un processeur de reconnaissance (23) est configuré pour reconnaître un identifiant de palette (25A) sur chacune des étiquettes de palette (31) capturées au sein du flux d'images (33) et lier l'identifiant de palette (25A) à l'une des baies pour palette (32).
PCT/SG2020/050743 2020-12-14 2020-12-14 Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur WO2022132025A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG2020/050743 WO2022132025A1 (fr) 2020-12-14 2020-12-14 Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2020/050743 WO2022132025A1 (fr) 2020-12-14 2020-12-14 Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur

Publications (1)

Publication Number Publication Date
WO2022132025A1 true WO2022132025A1 (fr) 2022-06-23

Family

ID=82059474

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2020/050743 WO2022132025A1 (fr) 2020-12-14 2020-12-14 Inventaire par drone autonome d'ensembles sur palettes placés à l'intérieur de baies pour palettes d'un entrepôt en intérieur

Country Status (1)

Country Link
WO (1) WO2022132025A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018057629A1 (fr) * 2016-09-20 2018-03-29 Foina Aislan Gomide Véhicules autonomes effectuant une gestion de stock
US20180094935A1 (en) * 2016-10-04 2018-04-05 Wal-Mart Stores, Inc. Systems and Methods for Autonomous Drone Navigation
US20190062055A1 (en) * 2017-08-28 2019-02-28 X Development Llc Robot Inventory Updates for Order Routing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018057629A1 (fr) * 2016-09-20 2018-03-29 Foina Aislan Gomide Véhicules autonomes effectuant une gestion de stock
US20180094935A1 (en) * 2016-10-04 2018-04-05 Wal-Mart Stores, Inc. Systems and Methods for Autonomous Drone Navigation
US20190062055A1 (en) * 2017-08-28 2019-02-28 X Development Llc Robot Inventory Updates for Order Routing

Similar Documents

Publication Publication Date Title
US20210379765A1 (en) Method for automatically generating waypoints for imaging shelves within a store
US11494830B1 (en) Determining an item involved in an event at an event location
US9505554B1 (en) Capturing packaging image via scanner
US10963835B1 (en) Monitoring locations within a materials handling facility
US11341454B2 (en) Method for tracking placement of products on shelves in a store
US11535451B2 (en) Autonomous robotic system for automatically monitoring the state of shelves in shops
US9280757B2 (en) Automated inventory management
US10388019B1 (en) Associating an agent with an event based on multiple inputs
US8396591B2 (en) System and method for data collection and analysis using robotic devices
CN101149792B (zh) 使用移动盘存机器人来执行盘存的系统和方法
US10296814B1 (en) Automated and periodic updating of item images data store
US9836651B2 (en) Displaying information relating to a designated marker
US20190197277A1 (en) Code recognition device
CN108805637B (zh) 自动关联格口并确认分播的方法和装置
CN108647553A (zh) 模型训练用图像的快速扩充方法、系统、设备及存储介质
WO2021046323A1 (fr) Inspection et gestion d'inventaire d'entrepôt par véhicule autonome
US20150066550A1 (en) Flow line data analysis device, system, non-transitory computer readable medium and method
US8762111B2 (en) Method for inputting a spatial layout of production devices to a computer-aided planning program and for optimizing the latter
JP2019040227A (ja) 在庫管理システム
US10552750B1 (en) Disambiguating between multiple users
US20110313734A1 (en) Model Assembly of a Production Plant With True-To-Scale Models of Production Devices and Method for Inputting a Spatial Assembly of Production Devices Into a Computer-Assisted Planning Program
JP2005320074A (ja) 物品探索収集装置およびプログラム
US11875570B1 (en) Updating agent position information
JP6967333B2 (ja) ピッキング支援システム及びピッキング支援方法
US11663742B1 (en) Agent and event verification

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20966107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20966107

Country of ref document: EP

Kind code of ref document: A1