CN113498524A

CN113498524A - Delivery system

Info

Publication number: CN113498524A
Application number: CN202080016751.6A
Authority: CN
Inventors: R·L·小马丁; K·马赫什; R·赫斯塔德; G·约翰; H·D·塔蒂尼; R·阿加瓦尔; J·C·米勒; R·拉古纳坦; J·梅伦德斯; D·佩德罗奇洛斯; C·伯登
Original assignee: Rehrig Pacific Co Inc
Current assignee: Rehrig Pacific Co Inc
Priority date: 2019-02-25
Filing date: 2020-01-31
Publication date: 2021-10-12
Also published as: WO2020176196A1; US20210331869A1; US20200273131A1; MX2021009905A; US11087160B2; CA3129719A1; EP4032049A1; US20220250845A1; US11046519B2; US11383930B2; US20210198042A1

Abstract

A delivery system generates a pick order containing a plurality of SKUs based on an order. Imaging a cargo-laden pallet to identify the SKU on the cargo-laden pallet, comparing the SKU to the order before the cargo-laden pallet leaves a distribution center. The cargo-laden pallet may be imaged while wrapped with stretch wrap. At the time of delivery, the pallet loaded with goods may again be imaged and analyzed for comparison with the picklists.

Description

Delivery system

Technical Field

The delivery of products from a distribution center to a store has many error-prone and inefficient steps. When an order is received from a customer, the specified product is loaded on at least one pallet according to a "pick list".

Background

For example, the product may be a beverage container box (e.g., a can carton and a beverage crate containing bottles or cans, etc.). There are many different permutations of styles, sizes and types of beverage containers delivered to each store. Missing or mis-picked products may result in significant additional operational costs when building pallets.

The pallets loaded with goods are then loaded on trucks along with pallets of other stores. A wrongly loaded pallet causes a significant time delay in the delivery route, as the driver will have to dispatch a possibly limited space in the trailer to rearrange the pallet during the delivery process. Additional pallets on the truck may also result in additional loading time to find the wrong pallet and reload it on the correct trailer.

At the shop, the driver unloads the goods from the pallet specified for the location. Drivers often spend a significant amount of time waiting at stores for store personnel to be available to register delivered products by physically counting the delivered products. During this process, the clerk ensures that all ordered products are delivered. Drivers and store personnel often break the pallet and open each box to scan one UPC per unique style and size. After scanning for unique styles and sizes, both the store clerk and the driver count the number of boxes or bottles for the UPC. This continues until all of the products on all of the pallets have been counted. Store personnel are often busy helping their own customers, forcing drivers to wait until the store personnel are available to register a product.

Disclosure of Invention

The improved delivery system provides improvements to several stages of the delivery process. While these improvements work well when practiced together, less than all or even any of these improvements can be practiced separately to obtain some benefit.

The improved delivery system facilitates order accuracy from warehouse to store by combining machine learning and computer vision software with serialized (RFID/barcode) shipping pallets. The pallet filling algorithm is based on product collocation and warehouse layout.

An electronic order accuracy check is performed when the pallet is built, loaded onto a trailer, and delivered to the store. When a pallet is built, the delivery system verifies the build to ensure that the correct product SKU is loaded on the correct pallet according to the pick list. Once the pallet is built, the overall computer visual SKU count for the particular pallet is compared to the pick list for the particular pallet to ensure that the pallet is properly built. This can be done before the pallet is stretch wrapped, thus reducing the cost of opening the pallet for auditing and correction. This also prevents shortages and excesses at the delivery point, thus preventing the driver from having to take back excess product or make additional trips to deliver the missing product.

The optimized queuing system can then be used to queue and load pallets onto the trailer in the correct reverse stop order (the last stop being loaded onto the trailer first). The handler will be able to see electronic visual controls showing which pallet will be loaded on which trailer, for example: a #3 pallet or the like is loaded in the #4 station area.

The system will also reduce the time for the recipient to register the product at the delivery point (e.g., store) by establishing a trusted combination of checks at the delivery point. This is accomplished by transporting the computer visual image of the verified SKU on the pallet before it leaves the warehouse and as it is delivered to the store. This may be a comparison of individual images, or deep machine learning by having the images at the store also electronically identify the product SKU. Delivery benefits include significantly reduced costs associated with waiting and checking products at the store level and verifiable electronic ledgers delivered for future review.

The delivery system will take one or more still images (e.g., 4, i.e., 1 on each side) of the pallet using the mobile device that the driver or recipient will own. The image may then be compared electronically to control pictures from the warehouse and physically compared by store personnel. The clerk can electronically sign all the product SKUs on their pick list. Different levels of receipts will be provided for clerk approval. The verification at the store may be a simple pallet serial scan via RFID/barcode and GPS coordinates for delivery, pallet image comparison, and/or SKU verification through machine learning computer vision algorithms invoked from the mobile device.

Drawings

Fig. 1 is a schematic view of a delivery system.

Fig. 2 shows an exemplary loading station of the delivery system of fig. 1.

Fig. 3 shows an exemplary verification station of the delivery system of fig. 1.

FIG. 4 is another view of the exemplary verification system of FIG. 3 with a pallet loaded with goods thereon.

FIG. 5 shows another exemplary verification system for the delivery system of FIG. 1.

Figure 6 shows the verification system of figure 5 during wrapping of a pallet carrying goods.

Fig. 7 shows yet another exemplary verification system of the delivery system of fig. 1.

Fig. 8 shows route optimization used in the delivery system of fig. 1.

Fig. 9 shows an exemplary loading station of the delivery system of fig. 1.

Fig. 10 is another view of the exemplary loading station of fig. 9.

Fig. 11 shows a scheduling system of the delivery system of fig. 1.

FIG. 12 illustrates a store notification feature of the delivery system of FIG. 1.

Fig. 13 is an exemplary screen of a mobile application for confirming a pallet ID in the delivery system of fig. 1.

FIG. 14 is an exemplary screen of a mobile application for imaging the cargo-laden pallet for verification in the delivery system of FIG. 1.

FIG. 15 is an exemplary screen of a mobile application in which a user may approve the image of the cargo board carrying cargo from FIG. 14.

FIG. 16 is a screen shot of an application on a mobile device indicating the number of each SKU that has been identified on the pallet carrying the cargo in the image of FIG. 15.

Fig. 17 shows an exemplary screen of a mobile application, where a driver has imaged a pallet carrying goods at a shop.

FIG. 18 shows an exemplary screen of a mobile application displaying confirmation that a SKU on a good-laden pallet at a store matches a pick order.

FIG. 19 shows another exemplary screen of a mobile application displaying confirmation that a SKU on a good-laden pallet at a store matches a pick order.

FIG. 20 shows an exemplary screen of a mobile application indicating that a SKU on a pallet with goods at a store does not match a pick order.

Fig. 21 shows an exemplary pallet sled with sensors and/or cameras for identifying pallets on the sled and items on the pallets.

Fig. 22 shows an exemplary training station of the delivery system of fig. 1.

Fig. 23 shows an alternative training station that may be used in the system of fig. 1.

FIG. 24 shows one possible architecture of the training features of the system of FIG. 1.

Fig. 25A and 25B are flow diagrams of one version of a method for delivering an item.

FIG. 26 is a flow diagram of one version of a method for training a machine learning model.

Fig. 27 shows an alternative authentication station.

Fig. 28 shows an exemplary screen indicating a verified cargo-laden pallet at a distribution center.

Fig. 29 shows an exemplary screen indicating a mispicked cargo-laden pallet at the distribution center.

Detailed Description

Fig. 1 is a high-level view of a delivery system 10 that includes one or more distribution centers 12, a central server 14 (e.g., a cloud computer), and a plurality of stores 16. A plurality of trucks 18 or other carrier vehicles each transport the products 20 on pallets 22 from one of the distribution centers 12 to a plurality of stores 16. Each truck 18 carries a plurality of pallets 22, which pallets 22 may be half-size pallets, each of which is loaded with a plurality of goods 20 for delivery to one of the stores 16. A wheeled trolley 24 is provided on each truck 18 to facilitate the delivery of one or more pallets 22 of goods 20 to each store 16. Generally, the cargo 20 may be loaded on a half-size pallet 22, a full-size pallet, a cart or trolley or truck, all considered herein as a "platform".

Each distribution center 12 includes one or more picking stations 30, each picking station 30 being associated with a verification station 32. Each verification station 32 is associated with a loading station 34, such as a loading station area for loading trucks 18.

Each distribution center 12 may have a plurality of loading stations 34. Each distribution center 12 includes a DC computer 26. The DC computer 26 receives the order 60 from the store 16 and communicates with the central server 14. Each DC computer 26 receives orders and generates a pick order 64, each of the pick orders 64 storing and associating a SKU with a pallet ID. Alternatively, the order 60 may be sent from the DC computer 26 to the central server 14 for generation of a pick order 64, the pick order 64 being synchronized back to the DC computer 26.

Some or all of the distribution centers 12 may include training stations 28 for generating image information and other information about new products 20 that may be transmitted to the central server 14 for analysis and future use.

The central server 14 may include a plurality of distribution center accounts 40 (including DC 1-DCn), each of which is associated with a distribution center 12. Each DC account 40 includes a plurality of store accounts 42, including store 1-store n. The order 60 and pickups 64 for each store are stored in the associated store account 42. The central server 14 further includes a machine learning model that includes a plurality of SKU files 44, including SKU 1-SKUn. The model is periodically synchronized to the DC computer 26.

The SKU files 44 each contain information for the SKU. A "SKU" may be a single variation of a product that is available from the distribution center 12 and can be delivered to one of the stores 16. For example, each SKU may be associated with a particular number of containers (e.g., 12) in a particular form (e.g., cans or bottles) having a particular packaging (cardboard or reusable plastic crates, etc.), having a particular style, and having a particular size (e.g., 24 ounces). This information is contained in each SKU file 44 along with the product name, product description, product size, and image information for the product. Each SKU file 44 may also include the weight of the product. The image information may be further decomposed into text and color information. It is also possible that more than one variation of a product may share a single SKU, for example where only the packaging, aesthetics and appearance of the product vary, but the contents and number are the same. For example, it is sometimes possible to utilize promotional packaging that will have different image information for a particular SKU. In general, all SKU files 44 including their associated image information may be generated by the training module 28.

Referring also to the flow chart in FIG. 25, in step 150, an order 60 may be received from the store 16. By way of example, the order 60 may be submitted by a store employee using an application or mobile device 52. The order 60 is sent to the distribution center computer 26 (or alternatively, to the server 14 and then relayed to the appropriate (e.g., nearest) distribution center computer 26). The distribution center computer 26 analyzes the order 60 and creates a pick order 64 associated with the order 60 in step 152. The pick order 64 assigns each of the SKUs (including the quantity of each SKU) according to the order. The pick order 64 specifies how many pallets 22 will be needed for the order (as determined by the DC computer 26). The DC computer 26 may also determine which SKUs should be loaded on the same pallet 22 in close proximity to each other, or if more than one pallet 22 would be needed, which SKUs should be loaded on the same pallet 22. For example, the SKUs entering the cooler may be collectively on the same pallet (or near each other on the same pallet), while the SKU placed on the shelf may be on another portion of the pallet (or, if there is more than one pallet, on another pallet). If the pick order 64 is created on the DC computer 26, it is copied to the server 14. If the pick order 64 is created on the server 14, it is copied to the DC computer 26.

Figure 2 shows the picking station 30 of figure 1. Referring to fig. 1 and 2, a worker at the distribution center reads the pallet ID (e.g., via RFID, bar code, etc.) on the pallet 22 on the pallet truck 24a, such as by means of a mobile device or reader on the pallet truck 24a (see screen shot of fig. 13). The shelf may contain, for each SKU, a plurality of items 20, such as a first product 20a for a first SKU and a second product 20b for a second SKU (collectively "products 20"). A worker reading a computer screen or mobile device screen displayed according to the pick order 64 retrieves each product 20 and places the product 20 on the pallet 22. Alternatively, the pallets 22 may be loaded by automated loading and unloading equipment.

In step 154, the worker places the item 20 on the pallet 22 according to the pick slip 64 and reports the pallet ID to the DC computer 26. The DC computer 26 provides for loading the items 20a, b on the pallet 22 in sales groups and subgroups for easier unloading at the store. In the illustrated example, the pick slip 64 provides that the products 20a are on one pallet 22 and the products 20b are on another pallet 22. For example, cooler items should be grouped and dry items should be grouped. The disassembly of the package group is also minimized to make unloading easier. This also makes the pallet 22 more stable.

After loading one pallet 22, the next pallet 22 is taken to the picking station 30 until all of the SKUs required for the order pick 64 are loaded onto as many pallets 22 as are required for the order pick 64. The pallet 22 is then loaded for the next pick order 64. The DC computer 26 records the pallet ID of the pallet 22 that has been loaded with the particular SKU for each pick order 64. The pick order 64 may associate each pallet ID with each SKU.

After loading, each pallet 22 loaded with goods is verified at a verification station 32, which verification station 32 may be adjacent to the picking station 30 or part of the picking station 30. As will be described in greater detail below, at least one still image, and preferably several still images or videos, of the products 20 on the pallet 22 are taken at the verification station 32 in step 156. The pallet ID of pallet 22 is also read. In step 158, the image is analyzed to determine the SKU of the product 20 currently on the identified pallet 22. In step 160, the DC computer 26 compares the SKUs of the products 20 on the pallet 22 to the order of pick 64 to ensure that all of the SKUs associated with the pallet ID of the pallet 22 on the order of pick 64 are present on the correct pallet 22 and that no additional SKUs are present. Several ways of carrying out the above steps are disclosed below.

First, referring to fig. 3 and 4, the verification station may include a CV/RFID semi-automatic wrapping machine 66a with a turntable 67, which may be specially equipped with a camera 68 and an RFID reader 70 (and/or a bar code reader). The wrap 66a holds a roll of translucent flexible plastic wrap or stretch wrap 72. As is known, the cargo-laden pallet 22 may be placed on a turntable 67, the turntable 67 rotating the cargo-laden pallet 22 as the stretch wrap 72 is applied. The camera 68 may be a depth camera. In this wrapping machine 66a, the camera 68 captures at least one image of the cargo-laden pallet 22 before or while the turret 67 is rotating the cargo-laden pallet 22 about the stretch wrap 72. Images/videos of the pallet 22 carrying the goods after wrapping may also be generated. As used herein, "image" or "images" generally refers to any combination of still images and/or video, and "imaging" means capturing any combination of still images and/or video. Also, preferably, 2 to 4 still images or videos are photographed.

In one embodiment, camera 68 is recording video (or continuously changing images), while turntable 67 is rotating. When camera 68 detects that the two outer ends of pallet 22 are equidistant (or that the side of pallet 22 facing camera 68 is perpendicular to the camera 68 view), camera 68 records a still image. The camera 68 may record four still images in this manner, one on each side of the pallet 22.

An RFID reader 70 (or bar code reader or the like) reads a pallet ID (unique serial number) from the pallet 22. The wrapping machine 66a includes a local computer 74 that communicates with the camera 68 and the RFID reader 70 a. The computer 74 may communicate with the DC computer 26 (and/or the server 14) via a wireless network card 76. The image and pallet ID are sent to the server 14 via the network card 76 and associated with the pick list 64 (fig. 1). Optionally, a weight sensor may be added to the turntable 67 and the known total weight of the product 20 and pallet 22 may be compared to the measured weight on the turntable 67 for confirmation. If the total weight on turntable 67 does not match the expected weight, an alarm is generated.

Alternatively, the turntable 67, camera 68, RFID reader 70 and computer 74 of fig. 3 and 4 may be used without a wrapping machine. Pallets 22 loaded with goods may be placed on the turntable 67 for verification only and may be subsequently wrapped manually or at another station.

Alternatively, referring to fig. 5 and 6, the verification station may include a camera 68 and RFID reader 70 (or bar code reader, etc.) mounted to the robotic wrapping machine 66 b. As is known, instead of holding the stretch wrap 72 stationary and rotating the pallet 22, the robotic wrapping machine 66b travels the stretch wrap 72 around the pallet 22 carrying the goods to wrap the pallet 22 carrying the goods. The robotic wrapping machine 66b includes a camera 68, an RFID reader 70, a computer 74, and a wireless network card 76.

Fig. 6 shows the robotic wrapping machine 66b wrapping the cargo-laden pallet 22 and items 20 with a stretch wrap 72 (as commonly used) and generating at least one image 62 of the cargo-laden pallet 22. The robotic wrapping machine 66b travels around the cargo carrying pallet 22 and generates at least one image 62 of the cargo carrying pallet 22 prior to and/or while wrapping the cargo carrying pallet 22. An image of the pallet 22 carrying the goods after wrapping may also be generated. The robotic wrapping machine 66b operates the same as the wrapping machine 66b of fig. 3 and 4, except for the fact that the robotic wrapping machine 66b travels around the stationary cargo-laden pallet 22.

Alternatively, referring to fig. 7, the verification station may include, for example, a worker having a networked camera on a mobile device 78 (e.g., a smartphone or tablet computer) for capturing one or more images 62 of a pallet 22 carrying the cargo prior to wrapping the pallet 22 carrying the cargo. Fig. 14 is a screen shot of an application on mobile device 78 that instructs the user to take two still images of the long side of pallet 22 carrying cargo (alternatively, the user may take a video while walking around pallet 22). Fig. 15 is a screen shot of an application on the mobile device 78 on which the user can approve images taken by the user. FIG. 16 is a screen shot of an application on the mobile device 78 indicating the number of products 20 per SKU that have been identified on the pallet 22.

Other means may be used to collect images of pallets loaded with goods. In any of the methods, image analysis and/or comparison to a pick list is performed on the DC computer 26 with a copy of the machine learning model. Alternatively, the analysis and comparison may be done on the server 14, on the local computer 74, or on the mobile device 78 or on another local networked computer.

As described above, the camera 68 (or a camera on the mobile device 78) may be a depth camera, i.e., it also provides distance information related to the image (e.g., pixel-by-pixel distance information or distance information for a pixel region). Depth cameras are known and utilize various techniques such as stereo vision (i.e., two cameras) or more than two cameras, time of flight or lasers, and the like. If a depth camera is used, it is easy to detect the edges of the products stacked on the pallet 22 (i.e., the edges of the entire stack and possibly the edges of individual adjacent products by detecting slight gaps or differences in adjacent angled surfaces). Likewise, the depth camera 68 may more easily detect when the cargo carrying pallet 22 is in a vertical plane from the view of the camera 68 to take a still image.

However, images of the pallet 22 with the cargo are collected, which are then analyzed in step 158 to determine the SKU of each item 20 on the pallet 22 (FIG. 25A). If applicable, the images and sizes of all sides of each possible product (including multiple versions of each SKU) are stored in server 14. If multiple still images or videos are collected, the known sizes of the pallet 22 and the items 20 are used to ensure that each item 20 is counted once, and only once. For example, first, multiple sides of pallet 22 carrying cargo may be identified in the image. The layers of the article 20 are then identified on each side. Individual articles 20 are then identified on each of the four sides of the cargo board 22 carrying the cargo.

The computer identifies the type of packaging for each article 20, such as a reusable beverage crate, a corrugated paper tray with a translucent plastic wrap, or a fully enclosed cardboard or carton. The computer also identifies the brand of each item 20 (e.g., a particular style from a particular manufacturer), for example, by reading images/text on the package. The package may be identified first, thus narrowing the list of possible brand options to be identified. Or vice versa, brands may be determined and used to narrow down possible packaging options to be identified. Alternatively, branding and packaging may be determined independently and then cross referenced for validation. In either approach, if one technique results in a recognition with more confidence, the result may be prioritized over the opposite recognition. For example, if a brand is determined with a low confidence and a package is determined with a high confidence and the identified brand is not available in the identified package, the identified package is used and then the next most likely brand available in the identified package is used.

After identifying individual items 20 on each of the four sides of the cargo board 22 carrying cargo, duplicate items are removed, i.e., it is determined which items are visible from more than one side and appear in more than one image, based on the known sizes of the items 20 and the cargo board 22. If some items are identified from one side with less confidence, but appear in another image where they are identified with more confidence, then the identification with more confidence is used.

For example, if the pallet 22 is a half-size pallet, it will be about 40 to about 48 inches by about 20 to about 24 inches, including 800 mm x 600 mm in metric system. The standard size beverage crates, beverage cartons and wrapped corrugated paper trays will all be visible from at least one side, most will be visible from at least two sides, and some will be visible on three sides.

If the pallet 22 is a full-size pallet (e.g., about 48 inches by about 40 inches, or 800 mm by 1200 mm), most of the product will be visible from one or both sides, but there may be some product that is not visible from either side. The size and weight of the concealed product can be determined by a rough comparison with the pick list. Alternatively, a stored image of the SKU (from the SKU file) that does not match the visible product may be displayed to the user, who may manually confirm the presence of the hidden product.

The computer vision generated SKU count for a particular pallet 22 is compared to the pick list 64 to ensure that the pallet 22 is properly constructed. This may be done before the pallet 22 with the cargo is wrapped, thus preventing opening of the pallet 22 for auditing and correction. If the constructed pallet 22 does not match the pick list 64 (step 162), the missing or wrong SKU is indicated to the worker (step 164), for example via a display (e.g., FIG. 29). The worker may then correct the item 20 on the pallet 22 (step 166) and resume validation (i.e., start a new image in step 156).

If the pallet 22 with the cargo is confirmed, positive feedback is given to the worker (e.g., fig. 28), and then the worker proceeds to wrap the pallet 22 with the cargo (step 168). After wrapping, additional images of the pallet 22 carrying the goods may be taken. For example, four images of a pallet carrying cargo may be taken before wrapping, and four other images of a pallet 22 carrying cargo may be taken after wrapping. All images are stored locally and sent to the server 14. The worker then moves the verified cargo-laden pallet 22 to the loading station 34 (step 170).

After the pallet 22 with the cargo has been verified, it is moved to a loading station 34 (FIG. 1). As explained in more detail below, at the loading station 34, the distribution center computer 26 ensures that the pallets 22 with cargo identified by each pallet ID are loaded onto the correct truck 18 in the correct order. For example, pallets 22 to be delivered at the end of the route are loaded first.

The computer (DC computer 26, server 14, or another computer) determines the efficient route traveled by each truck 18 to visit each store 16 in the most efficient order, the specific cargo-laden pallets 22 that must enter each truck 18, and the order in which the pallets 22 should be loaded onto the trucks 18.

As shown in FIG. 8, the server 14 optimizes the route for each truck 18 so that an efficient route is drawn for the driver. As shown, the route is communicated to the driver's mobile device 50 (or in-vehicle navigation system) and may be modified as needed (e.g., based on traffic) after the truck 18 has left the DC 12.

Referring to fig. 9, an optimized queuing system is used to queue and load pallets 22 loaded with cargo onto truck 18 (the last stop is loaded first onto truck 18) in the correct reverse stop order based on the route planned for truck 18. Each truck 18 will be at a different loading station area doorway 80.

Fig. 10 shows an exemplary loading station 34, such as a loading station area (dock) having a doorway 80. Based on the sequence determined by the server 14, the electronic visual display 82 near the doorway 80 displays which pallet 22 is next to be loaded onto the truck 18. The cameras 84 and/or RFID readers 86 adjacent the doorway 80 identify each cargo board 22 carrying cargo as it is loaded onto the truck 18. If the wrong pallet 22 is moved toward the doorway 80, an audible and/or visual alarm alerts the worker. Alternatively, the RFID reader 86 at the doorway 80 can determine the direction of movement of the RFID tag on the cargo board 22 carrying the cargo, i.e., it can determine whether the cargo board 22 carrying the cargo is moving onto the truck 18 or away from the truck 18. This may be helpful if the wrong cargo-laden pallet 22 is moved onto the truck 18. The worker is notified that the wrong pallet 22 is loaded and the RFID reader 86 can confirm that the pallet is then moved away from the truck 18.

When a group of cargo-laden pallets 22 (two or more) are to go to the same store 16, the cargo-laden pallets 22 within the group may be loaded onto the truck 18 in any order. The display 82 may indicate that the group of pallets 22 carrying goods and the pallets 22 carrying goods within the group destined for the same store 16 will be approved by the RFID reader 86 and the display 82 in any order within the group.

Referring to FIG. 11, portal site 88 (generated by server 14) provides visibility of truck 18 scheduling for local companies to reduce latency.

Referring to fig. 1, a cargo-laden truck 18 carries a hand truck or pallet block 24 for moving a cargo-laden pallet 22 off of the truck 18 and into the store 16 (fig. 25, step 172). The driver has a mobile device 50 that receives the optimized route from the distribution center computer 26 or the center server 14. The driver follows the route to each of the plurality of stores 16 for which the truck 18 includes a pallet 22 carrying cargo.

At each store 16, the driver's mobile device 50 indicates which pallets 22 with goods (based on their pallet ID) are to be delivered to the store 16 (as evidenced by gps on the mobile device 50). The driver confirms the correct pallet for the location by means of the mobile device 50 checking the pallet ID (RFID, barcode, etc.). The driver moves the pallet 22 with the goods loaded therein into the store 16 by means of the pallet carriers 24.

Referring to FIG. 21, the pallet trolley 24 may optionally include an RFID reader 90 to check the pallet ID of the pallet 22 carried thereon by reading an RFID tag 94 affixed to the pallet 22. The RFID reader 90 may also read an RFID tag 96 on the item 20. Alternatively, or in addition, the pallet block 24 may include a camera 92 for imaging the cargo-laden pallet 22 carried thereon for verification. A local wireless communication circuit (e.g., bluetooth) may communicate the pallet ID of the pallet 22 on the pallet block 24 to the driver's mobile device 50. The driver's mobile device 50 may confirm to the driver that the correct pallet 22 is loaded on the pallet block 24 or alert the driver when the pallet 22 on the pallet block 24 does not correspond to the store 16 at the current location (determined via gps on the mobile device 50).

The pallet sled 24 may also help track the return of pallets 22 and returnable packages (e.g., plastic beverage crates 98). If the returnable package (e.g., plastic beverage crate 98) has an RFID tag 96, the pallet skid 24 may count the number of crates 98 and pallets 22 returned to the truck 18. Over time, this may provide asset tracking information. This makes it easy to determine, for example, whether the number of pallets 22 and crates 98 delivered to a particular store 16 consistently exceeds the number of pallets 22 and crates 98 returned from that store 16, thus indicating that the store 16 is experiencing a high rate of asset loss for some reason, which may then be investigated and remedied.

One of several methods may then be followed at the store.

In a first method, a driver removes the wrapper from the cargo carrying pallet 22 and uses the mobile device 50 to take at least one, and preferably several, still images or videos of the cargo carrying pallet 22 in the store 16 (fig. 1, 17, 25, step 174). Alternatively, the driver may be able to take a single image of a corner of an open cargo-laden pallet 22 such that both sides of the cargo-laden pallet 22 are captured in the single image. The image 62 is sent from the mobile device 50 to the server 14 (or alternatively, the DC computer 26). In step 176, the distribution center server 14 analyzes the images in one of the several ways described above to confirm the presence of the correct number of items 20 for each of the SKUs associated with the pallet ID of the pallet 22 on the pick slip 64 (step 178), and then communicates the confirmation to the driver's mobile device 50 and/or the store employee's mobile device 52, which is displayed on the screen (FIGS. 18 and 19).

If a discrepancy is detected (step 180), then in step 182 the system indicates to the driver the particular discrepancy and how to remedy the discrepancy. The driver may correct the discrepancy by retrieving the product 20 of the missing SKU from the truck 18 or crediting the store account 42 with the missing SKU (step 184). Any SKUs detected as not belonging to the pallet 22 may be returned to the truck 18 by the driver. On the store worker's mobile device 52 (via the application), the store worker confirms the presence of the pallet 22 carrying the good and receives the SKU list associated with the pallet ID from the distribution center computer 26 or the server 14.

If one or more of the SKUs do not match, the screen of FIG. 20 is displayed to the driver specifically indicating what is missing (step 182). Optionally, not shown, the screen on his mobile device may also visually indicate on the image the SKU that does not match, for example by drawing a box or circle around the SKU in the image. He can manually identify it by clicking on it and then assigning it the correct SKU, if necessary. If the SKU is actually physically lost or validly not on the pick list, it will also be identified here, and the driver can potentially correct the order, for example, by retrieving the missing items from the truck 18 in step 184. The driver then completes the delivery in step 186.

Referring to fig. 12, a store employee may receive a notification via their mobile device 52 that a delivery has been made. Via its mobile device 52, the employee may view the image of the pallet 22 carrying the goods, and may be required to sign the delivery based on the image and/or based on an indication from the server 14 that the system 10 has confirmed the accuracy of the delivery (i.e., after verifying the in-store image).

In a second method, the driver images the cargo-laden pallets 22 before opening them (again, one or more still images or videos of each cargo-laden pallet 22). The image 62 is transmitted from the mobile device 50 to the distribution center computer 26 or server 14. The distribution center computer 26 or central server 14 analyzes the image by identifying SKUs through the translucent stretch wrap. Alternatively, instead of a complete new identification of the SKU on the cargo carrying pallet 22, all that is required is to confirm that nothing has changed on the previously verified cargo carrying pallet 22. For example, knowing the previous arrangement of each SKU on the pallet 22 and the specific packaging of each SKU (for SKUs that may have more than one possible packaging), it is easier to identify those SKUs that are still in the same location and arrangement as when they were verified at the DC 12.

Additionally, the DC computer 26 and/or server 14 may also verify that the wrap is relatively undisturbed if images of the pallet 22 carrying the cargo are also taken after wrapping. Alternatively, it may be determined that the wrap is undisturbed without identifying the SKU below, and if the wrap is too disturbed, the driver is notified to remove the wrap and image the cargo-laden pallet 22 opened for full image analysis. Again, the store worker confirms the presence of the pallet 22 carrying the goods and receives the SKU inventory associated with the pallet ID from the distribution center computer 26 or the server 14.

Alternatively, the image may simply be compared as an image to an image taken at the distribution center without actually identifying the SKU in the image. The accuracy of the delivery can be confirmed if the image at the store is sufficiently similar to the image taken at the time of verification. This can be done by comparing the opened images to each other or by comparing the wrapped images to each other. However, this would not enable the driver to easily correct the missing SKU. Thus, if it is determined that the image is sufficiently similar to the verification image, new SKU identification based on the image of the open cargo-laden pallet 22 at the store 16 may begin at this point.

In a third method, the store worker has trusted the entire system 10 and simply confirms that the pallet 22 with the cargo has been delivered to the store 16 without spending time checking and comparing each SKU against its ordered list on a SKU-by-SKU basis and without requiring any re-verification/imaging by the driver. In this manner, the driver may immediately begin to remove products 20 from the pallet 22 and place them on the shelves 54 or in the cooler 56 as appropriate. This greatly reduces the delivery time for the driver.

FIG. 22 shows a sample training station 28 that includes a turntable 100 upon which a new product 20 (e.g., for a new SKU or a new variation of an existing SKU) can be placed to create a SKU file 44. The turntable 100 may include an RFID reader 102 for reading the RFID tags 96 (if present) on the products 20 and a weight sensor 104 for determining the weight of the products 20. As the product 20 rotates on the turntable 100, the camera 106 takes a plurality of still images and/or videos of the packaging of the product 20, including any markings 108 or any other indicia on the packaging. Preferably, all sides of the package are imaged. The image, weight, RFID information is sent to the server 14 for storage in the SKU file 44. Optionally, multiple images of the product 20 are taken at different angles and/or with different lighting. Alternatively or additionally, computer files having illustrations of the packaging for the product 20 (i.e., the files from which the packaging was made) are sent directly to the server 14.

Fig. 23 shows an alternative training station 28a that may be used in the system of fig. 1. The training station 28a includes a rack 120 on which new products 20 (e.g., for new SKUs or new variants of existing SKUs) can be placed to create the SKU file 44. The stand 120 may include an RFID reader 102 for reading the RFID tags 96 (if present) on the products 20 and an optional weight sensor 104 for determining the weight of the products 20. The one or more cameras 126 take a plurality of still images and/or videos of the packaging of the product 20, including any markings 108 or any other indicia on the packaging. In the example shown, three cameras 126 are mounted to a frame 128 that is fixed to the stand 120. Preferably, all sides of the package are imaged. Thus, in the illustrated example, after capturing three sides with three cameras 126, the user may rotate the product 20 so that the remaining three sides may be captured. The image, weight, RFID information may be received by the local training computer 130 and sent to the server 14 for storage in the SKU file 44. Again, optionally, multiple sets of images may be taken with different illuminations.

Each of the

cameras

106 or 126 may be a depth camera, i.e., it also provides distance information related to the image (e.g., pixel-by-pixel distance information or distance information for a pixel region). Depth cameras are known and utilize various techniques such as stereo vision (i.e., two cameras) or more than two cameras, time of flight or lasers, and the like. If a depth camera is used, it is easy to detect the edges of the product 20.

In one possible embodiment of either

training station

28 or 28a shown in FIG. 24, the cropped images of the product 20 from the training station 28 are sent from the local computer 130 to the SKU image storage 134 via the portal site 132, and the SKU image storage 134 may be at the server 14. Alternatively, a computer file with an illustration of the packaging for the product 20 (i.e., the file from which the packaging was made) is sent directly to the server 14.

Regardless of the method used to obtain the image of the item, the image of the item is received in step 190 of FIG. 26. In step 192, the API 136 acquires and builds the SKU images into a plurality of virtual pallets, each of which displays the appearance of the product 20 on the pallet 22. The virtual pallet may include four or five layers of products 20 on a pallet 22. Some of the virtual pallets may consist of only a single new product 20 and some of the virtual pallets will have a mix of images of different products 20 on the pallet 22. The API 136 also automatically marks the location and/or boundaries of the products 20 on the virtual pallet with the associated SKUs. The API creates a plurality of configurations of virtual pallets to send to the machine learning model 138 in step 194 to update it with the new SKU and picture.

The virtual pallet is constructed based on a set of configurable rules including the size of the pallet 22, the size of the products 20, the number of permitted levels (e.g., four levels, but five or six levels are also possible), level limits as to which products may be on which levels (e.g., certain bottles may only be on the top level), and so forth. The image of each virtual pallet is sized to be a constant size (or at least within a certain range) and placed on a virtual background (e.g., a warehouse scene). There may be multiple virtual backgrounds available from which to choose randomly.

The virtual pallet image is sent to the machine learning model 138 with a bounding box indicating the boundaries of each product on the image and the SKU associated with each product. The virtual pallet image, together with the bounding box and associated SKU, constitute training data for the machine learning model.

In step 196, the machine learning model 138 analyzes the image of the virtual pallet based on the location, boundary, and SKU tag information. The machine learning model 140 is updated and stored. The machine learning model 142 is deployed and used in conjunction with the verification station 32 (FIG. 1) and optionally the delivery methods described above. The machine learning model 138 may also receive actual images taken in the distribution center or store, which may be added to the machine learning model after recognition. Optionally, feedback from the worker may consider whether to use the image, e.g., not use the identified image, until the user has an opportunity to confirm or disprove the identification.

Fig. 27 shows another alternative authentication station. Pallets 22 loaded with goods 20 are transported on first conveyor 220 to turret 267. An RFID reader 270 and at least one depth camera 268 are positioned adjacent to the turntable 267. When the pallet 22 with cargo arrives at the turntable 267, the RFID reader 270 identifies the pallet 22 and the pallet 22 with cargo rotates on the turntable 267 so that the camera 268 can take images or video (as previously described), such as one still image of each of the four sides of the pallet 22 with cargo. As previously described, images are used to identify all SKUs on the pallet 22, which are compared to the pick list associated with the pallet 22. If the loaded pallet 22 is verified against the pick list, the loaded pallet 22 is moved to the second conveyor 222 and the second conveyor 222 carries the loaded pallet 22 to a dedicated wrapping station having a turret 267 and stretch wrap 272. At the wrapping station, the pallet 22 with the goods wrapped is wrapped with stretch wrap. If the pallet 22 with the load is not verified against the pick list, the pallet 22 with the load is moved on the third conveyor 224 to an audit station 226 where the worker can correct the load 20 on the pallet 22 in the manner explained above in other embodiments.

In accordance with the provisions of the patent statutes and law, the exemplary configuration set forth above is considered to represent a preferred embodiment of the present invention. It should be noted, however, that the invention may be practiced otherwise than as specifically illustrated and described without departing from its spirit or scope. Alphanumeric identifiers on method steps are for convenience only to refer to in the dependent claims and do not by themselves indicate the required order of implementation unless explicitly stated otherwise.

Claims

1. A method of delivery, comprising:

(a) receiving an order for a plurality of SKUs;

(b) generating a pick order based on the orders for the plurality of SKUs;

(c) configuring a plurality of items based on a pick order;

(d) imaging the assembled plurality of articles to generate at least one image;

(e) analyzing the at least one image to identify SKUs for the assembled plurality of items;

(f) comparing the SKU identified in step (e) with SKUs on a pick-up order; and

(g) indicating whether the SKU identified in step (e) matches a SKU on the order based on the comparison in step (f).

2. The method of claim 1, wherein in step (c), a plurality of articles are assembled on a platform.

3. The method of claim 2, wherein the platform is a pallet.

4. The method of claim 3, further comprising the steps of:

(h) after said steps (a) - (g), moving the pallet with the goods to a store associated with the order; and

(i) after said step (h), unloading the goods from the pallet loaded with goods at the store.

5. The method of claim 4, further comprising the steps of:

(j) after said steps (a) - (h), imaging the cargo board carrying the cargo at the store to generate at least one store image; and

(k) after said step (j), analyzing at least one store image to confirm the validity of the pallet carrying the goods.

6. The method of claim 5, wherein step (k) further comprises:

(l) Analyzing the at least one store image to identify a SKU for the item on the pallet; and

(m) comparing the SKU identified in step (l) with SKUs on a pick-up order.

7. The method of claim 6, further comprising the steps of:

(n) in said step (g), indicating the absence of SKUs from the pickorders on the pallet; and

(o) placing the missing SKUs on the pallet after said step (n).

8. The method of claim 6, further comprising the steps of:

(n) after step (c) and before step (h), placing a wrap around the pallet carrying the goods.

9. The method of claim 8, further comprising the steps of:

(o) after step (h) and before said step (j), removing the wrapper from around the cargo-laden pallet.

10. The method of claim 9, wherein step (d) is carried out by a camera mounted to a wrapping machine carrying the wrapper.

11. The method of claim 10, wherein said step (d) is performed during said step (n).

12. The method of claim 3, wherein the order is a first order of a plurality of orders received from a plurality of stores, wherein the pallet is one of a plurality of pallets, the method further comprising:

(h) assigning each order of a plurality of orders to one delivery route of a plurality of delivery routes, each delivery route of the plurality of delivery routes to be covered by one truck of a plurality of trucks;

(i) determining, for each delivery route of a plurality of delivery routes, an order in which stores along each delivery route are to be visited;

(j) an order of loading the pallets onto each of the plurality of trucks is determined based on the associated delivery routes.

13. The method of claim 12, further comprising the step of:

(k) identifying each of the plurality of pallets as they are loaded onto the plurality of trucks; and

(l) Generating feedback based on step (k) compared to the order determined in step (j).

14. The method of claim 13, wherein step (k) includes the step of reading the RFID on each pallet as it approaches the loading bay.

15. The method of claim 1, further comprising the steps of:

(h) imaging the new SKU to generate a plurality of images of the new SKU; and

(i) adding the plurality of images of the new SKU to the database so that the new SKU can be identified in step (e).

16. A verification system, comprising:

a wrapping machine for placing a wrapping around a platform loaded with items each having an associated SKU; a camera mounted to the wrapping machine, the camera configured to image the cargo-laden platform prior to or during wrapping of the cargo-laden platform; and

a computer programmed to analyze the image generated by the camera to identify a SKU for the item on the platform.

17. A verification system according to claim 16 wherein said wrapping machine includes a turntable for receiving and rotating a platform carrying cargo thereon during wrapping thereof.

18. A verification system as claimed in claim 17 wherein the wrapping machine includes an RFID reader for reading an RFID tag on the platform when the platform is on the turntable.

19. A verification system according to claim 16 wherein the wrapping machine is configured to travel around the cargo-laden platform in a roll of wrapping for wrapping around the cargo-laden platform.

20. A verification system as claimed in claim 19 wherein the wrapping machine includes an RFID reader for reading an RFID tag on a platform to be wrapped.

21. A method for delivery verification, comprising the steps of:

(a) bringing a plurality of items to a store in response to an order;

(b) imaging the plurality of items after step (a) to generate at least one store image;

(c) analyzing the at least one store image to determine SKUs for the plurality of items;

(d) comparing the SKU determined in step (c) with the order; and

(e) indicating whether the SKUs for the plurality of items match the order.

22. The method of claim 21, wherein the plurality of items in step (a) are on a pallet, and wherein step (b) comprises imaging a plurality of sides of the plurality of items, and wherein the at least one store image is a plurality of store images; and wherein step (c) comprises the steps of: the method includes determining a layer of a plurality of items on a pallet in each of a plurality of store images, determining a SKU of the items visible in each of the plurality of store images, and removing duplicate items appearing in more than one of the plurality of images.

23. The method of claim 21, further comprising the step of removing the wrapper around the plurality of items prior to step (b).

24. The method of claim 21, wherein the plurality of items are containers of beverage containers.

25. A method for adding an item to a delivery verification database, comprising the steps of:

(a) imaging the new item to generate a plurality of images of the new item; and

(b) adding multiple images of the new item to the database enables the new item to be identified on the pallet.

26. The method of claim 25, further comprising the step of imaging a plurality of sides of the new item.

27. The method of claim 26, further comprising the step of identifying text and color in a plurality of images of a new item.

28. The method of claim 26, further comprising the step of generating a virtual heap of multiple copies of the new item, analyzing the virtual heap in a machine learning module.

29. A method of training a machine learning process, comprising the steps of:

(a) generating virtual images of a plurality of items on a platform; and

(b) virtual images of a plurality of items on a platform are processed in a machine learning model to train the machine learning model.

30. The method of claim 29, further comprising the step of indicating a boundary of an item in a virtual image and indicating a SKU associated with the boundary.

31. The method of claim 30, further comprising the step of complying with a set of constraints on generating virtual images.

32. The method of claim 31, wherein the virtual image is generated based on a computer file used to create a package of the item.