WO2023028507A1 - Système de suivi d'actif - Google Patents

Système de suivi d'actif Download PDF

Info

Publication number
WO2023028507A1
WO2023028507A1 PCT/US2022/075377 US2022075377W WO2023028507A1 WO 2023028507 A1 WO2023028507 A1 WO 2023028507A1 US 2022075377 W US2022075377 W US 2022075377W WO 2023028507 A1 WO2023028507 A1 WO 2023028507A1
Authority
WO
WIPO (PCT)
Prior art keywords
asset
sensor data
data
determining
identity
Prior art date
Application number
PCT/US2022/075377
Other languages
English (en)
Inventor
Ashutosh Prasad
Vivek Prasad
Original Assignee
Koireader Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koireader Technologies, Inc. filed Critical Koireader Technologies, Inc.
Publication of WO2023028507A1 publication Critical patent/WO2023028507A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management

Definitions

  • Storage facilities such as shipping yards, processing plants, warehouses, distribution centers, ports, yards, transports, and the like may store vast quantities of assets over a period of time. As the assets are moved and transferred the assets may become lost or damaged. Monitoring the assets is typically a manual task performed as part of weekly, monthly, and yearly audits at each location. These audits are often time consuming and may be prone to errors. Additionally, between audits, assets may be lost or otherwise misplaced resulting in logistical delays and the like.
  • FIG. 1 is an example block diagram of an asset management system for providing asset tracking capabilities and other security features.
  • FIG. 2 is a flow diagram illustrating an example process associated with determining a change in custody of an asset according to some implementations.
  • FIG. 3 is a flow diagram illustrating an example process associated with determining a change in custody of an asset according to some implementations.
  • FIG. 4 is a flow diagram illustrating an example process associated with determining damage to an asset according to some implementations.
  • FIG. 5 is another flow diagram illustrating an example process associated with determining damage to an asset according to some implementations.
  • FIG. 6 is another flow diagram illustrating an example process associated with determining damage to an asset according to some implementations.
  • FIG. 7 is a flow diagram illustrating an example process associated with providing asset security according to some implementations.
  • FIG. 8 is an example sensor system that may implement the techniques described herein according to some implementations.
  • FIG. 9 is an example asset management system that may implement the techniques described herein according to some implementations.
  • FIG. 10 is an example pictorial view associated with the systems of FIGS. 1-9 according to some implementations.
  • FIG. 11 is another example pictorial view associated with the systems of FIGS. 1-9 according to some implementations.
  • FIG. 12 is another example pictorial view associated with the systems of FIGS. 1-9 according to some implementations.
  • an asset management system may include, but not be limited to, an inventory management system, a chain of custody system, a warehouse management system, an asset management system, facility management system, a supply chain management system, a container inventory management system, and/or the like.
  • the asset management system may be a remote or cloud-based system that is communicatively coupled to a plurality of sensor systems located at various facilities, vehicles, containers, and the like and may include on site equipment such as sensors (including but not limited to image devices, thermal sensors, proximity sensors, motion sensors, and the like), edge computing system, servers, and the like.
  • sensors including but not limited to image devices, thermal sensors, proximity sensors, motion sensors, and the like
  • edge computing system servers, and the like.
  • a sensor systems may be associated with an entry location and/or an exit location of a facility as well as within a transport’s cargo area.
  • the sensor system may capture sensor data of an asset (e.g., a transport handling unit, package, inventory item, and the like) being loaded and/or unloaded from a transportation unit (such as a truck, shipping container, airfreight container, vehicle hold, and the like).
  • the sensors may be configured to detect identifiers, such as RFID UWB or BLE tags, QR codes, bar codes, alpha/numerical codes, text, logos, symbols (like fragile etc.) and the like, associated with assets.
  • the asset system may determine if the assets are correctly loaded/unloaded (e.g., placed or stacked correctly and actually loaded/unloaded), verify or confirm an identity of the operator that loaded/unloaded the assets.
  • the system may also determine based on the sensor data if an asset is open or otherwise tampered with prior to loading/unloading.
  • the asset management system may also count the number of transport handing units (THU), pallets, assets, cartoons, units, or the like as they are loaded/unloaded from a transport, THU, pallet or even unboxed.
  • THU transport handing units
  • the system may detect a mislabeling, incorrect identify, or misplaced identifier (e.g., outside of threshold boundary location) of an asset. For example, if an individual is stickering the assets prior to shipping, the system may determine if one or more of the assets were missed, if the wrong sticker was applied to the asset, if the sticker was misplaced or mislocated on the asset (e.g., not at an expected location on the asset), and the like.
  • a mislabeling, incorrect identify, or misplaced identifier e.g., outside of threshold boundary location
  • the asset system may also transfer a chain of custody associated with the assets.
  • the asset system may notify the asset owners (e.g., buyers and/or sellers), responsible parties (e.g., facility owners, transport owners, and the like), insurance carriers associated with the assets, and the like.
  • the asset management system may accurately assign lability in the case of damage, pilferage, and/or other issues concerning the assets over the lifecycle of the assets.
  • the sensors within a facility may be mounted with a field of view of a dock, exit, and/or entrance to the facility.
  • the sensors may be placed below (e.g., along the floor), above (e.g., along the ceiling), or adjacent to the dock doors (e.g., to the right and/or left of the exit/entrance).
  • the asset system may parse or extract from sensor data (such as image data) information usable to determine the identity of the asset.
  • the asset system may receive two-dimensional (2D) and/or three-dimensional (3D) image data of the asset that includes an identifier, such as a tag, label, or the like.
  • the asset system may utilize the sensor data to determine an identity based on an estimated size, shape package or carton count, and the like.
  • a machine learned model or system may be trained on various types of assets such that the machine learned model or system may determine the identity based on various characteristics or features of each asset without relying on, for instance, solely on the identifiers.
  • the transports may also be equipped with sensors located within the cargo area and along the walls adjacent to the entrance to the cargo area (e.g., above, below, to the left and/or right of the opening).
  • the asset system may track the asset positions within the facility even when the identifier is periodically lost from view of the sensors and moved between entities (e.g., facilities and transports). If the asset is lost, the asset management system may estimate a position of the asset during the period of which the asset is lost from view and then update the position data upon re- -detection of the asset via the identifier.
  • exits and entrances to facilities are non-uniform and the loading and unloading are performed by humans in a non-organized maimer.
  • the assets may be tightly stacked such that the sensor systems within the transport are unable to capture data associated with a particular asset.
  • the assets may be removed at various different orientation and angles. For example, an individual may pick up and hold the asset in front of their body/arms, on their shoulder, under either of their arms, and the like. In this manner, the sensor data may only have partial or temporary data representing the identifier. In these cases, the system may identify the asset from the temporary visibility.
  • the system may then identify and track the individual or other data related to the asset as well as estimate the trajectory and/or path of the asset as the individual carries the asset into a facility.
  • the individual may, in some cases, place the asset in a manner that the identity cannot be confirmed. For instance, the asset may be placed or stacked with the identifier facing the ground or another asset.
  • the system may estimate the position/location of the asset.
  • the system may generate a notification to one or more responsible parties providing the information related to the last known position and location and the estimated position or location.
  • the system may determine a custodial responsibility at the time the asset was last identified. The system may then notify the responsible party as well as an appropriate or associated insurance carrier such that a claim can be opened. The system may also identify the harmed party (such as the seller or buyer) such that the claim may be compensated by the insurance carrier.
  • a THU may include, but is not limited to, pallets, bins, unit load devices (ULDs), ocean containers, any object that may carry or otherwise transport an inventory item, and the like.
  • ULDs unit load devices
  • the originating facility may have failed to place the asset in the container, package, or THU.
  • the destination facility may have miscounted the assets as the assets are unloaded.
  • a transport operator or other authorized individual may access the assets during transport and remove one or two assets from a container, package or THU, reseal the container, package or THU, and complete delivery.
  • the system discussed herein tracks the assets within the originating facility (during picking, packaging, and loading) until the assets are transferred to the transport (during transport), and at the destination facility (during unload, unboxing, and the like) as well as estimating positions when the assets is not visible to the sensors, the system may assign responsibility for the missing items.
  • the system may determine the responsibility is the originating facility. If the asset was accessed during transport, the system may determine the responsibility is the transport company. And if the asset was lost after delivery, the system may determine the responsibility is the destination facility. In some cases, the system may also include location data (such as GPS data, Glonass data, GNSS data, WiFi network position data, and the like) and may, upon a determination that a THU or package is being accessed, notify law enforcement or other government bodies that a pilferage is in process and provide the transport identification, the asset identification, image data associated with the perpetrator, and any information known about the perpetrator (such as when the perpetrator is the driver).
  • location data such as GPS data, Glonass data, GNSS data, WiFi network position data, and the like
  • the pilferage may be prevented prior to the item becoming lost.
  • the system may also notify an operator of an originating facility, when the system determines the package or THU has been misloaded prior to the package or THU being loaded on the transport. In this manner, the system may not only more accurately assign custody and responsibility with respect to insurance claims, but also prevent or otherwise reduce the number of insurance claims initiating from the packaging, storage, and transportation industries.
  • the asset management system may process the sensor data using one or more machine learned models to identify the assets, track and/or estimate position data associated with packages, vehicles or individual transporting the assets, and the like.
  • the machine learned models may be generated using various machine learning techniques.
  • the models may be generated using one or more neural network(s).
  • a neural network may be a biologically inspired algorithm or technique which passes input data (e.g., image and sensor data captured by the loT (Internet of Things) computing devices) through a series of connected layers to produce an output or learned inference.
  • Each layer in a neural network can also comprise another neural network or can comprise any number of layers (whether convolutional or not).
  • a neural network can utilize machine learning, which can refer to a broad class of such techniques in which an output is generated based on learned parameters.
  • one or more neural network(s) may generate any number of learned inferences or heads from the captured sensor and/or image data.
  • the neural network may be a trained network architecture that is end-to-end.
  • the machine learned models may include segmenting and/or classifying extracted deep convolutional features of the sensor and/or image data into semantic data.
  • appropriate truth outputs of the model in the form of semantic per-pixel classifications and recognition (e.g., vehicle identifier, container identifier, driver identifier, and the like).
  • machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least-angle regression (LARS)), decisions tree algorithms (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), Chi-squared automatic interaction detection (CHAID), decision stump, conditional decision trees), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average one-dependence estimators (AODE), Bayesian belief network (BNN), Bayesian networks), clustering algorithms (e.g., OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated
  • MDS Projection Pursuit
  • LDA Linear Discriminant Analysis
  • MDA Mixture Discriminant Analysis
  • QDA Quadratic Discriminant Analysis
  • FDA Flexible Discriminant Analysis
  • Ensemble Algorithms e.g., Boosting, Bootstrapped Aggregation (Bagging), AdaBoost, Stacked Generalization (blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.
  • Additional examples of architectures include neural networks such as ResNet50, ResNetlOl, VGG, DenseNet, PointNet, and the like.
  • the system may also apply Gaussian blurs, Bayes Functions, color analyzing or processing techniques and/or a combination thereof.
  • the sensor system may include one or more loT devices.
  • the loT computing devices may include a smart network video recorder (NVR) or other type of EDGE computing device with a GPU/NPU/CPU.
  • NVR smart network video recorder
  • Each loT device may also be equipped with sensors and/or image capture devices, such as visible light image systems, infrared image systems, radar based image systems, LIDAR based image systems, S WIR based image systems, Muon based image systems, radio wave based image systems, and/or the like.
  • the loT computing devices may also be equipped with models and instructions to capture, parse, identify, and extract information associated with a lifecycle of an asset, as discussed herein, in lieu of or in addition to the cloud-based services.
  • the loT computing devices and/or the cloud-based services may be configured to perform segmentation, classification, attribute detection, recognition, data extraction, and the like.
  • FIG. 1 is an example block diagram 100 of an asset management system 102 for providing asset tracking capabilities and other security features.
  • the asset management system 102 may receive sensor data 104 from various devices, generally indicated by 106.
  • the devices 106 may include one or more loT devices or sensors installed at fixed locations throughout or systems associated with an origination facility, destination facility, and/or a transport, such as a container, ship, plane, truck, train, or the like.
  • the sensor system 106 may be associate with a conveyor or other robotic or autonomous system within the facility (e.g., an assembly or packing arm or the like).
  • the sensor system 106 may include image devices, recording and data storage devices or systems, as well as gyroscopes, accelerometers, inertial measurement units (IMUs), location systems (such GPS, Glonass, GNSS, and the like, and the like.
  • the sensors 106 may collect data along with the image or video data during pickup, put away, replenishment, loading, unloading, unpacking, and the like.
  • the image or video data may be sent to an EDGE computing device or cloud-based service over a wireless interface (such as streaming data, streaming real-time data, and the like to a remote system for processing) to generate an estimated position and location data for each asset being tracked.
  • the sensor data 104 may include image data associated with a field of view of the sensor 106 associated with the implement of a vehicle such as a, forklift.
  • the sensors 106 may be associated with a fixed position of the facility, worn by an operator of the facility, positions along a conveyor belt or work area, affixed on a robotic system, and the like.
  • the asset management system 102 may determine an identity of an asset, a THU, or the like based at least in part on one or more identifiers within the field of view of the sensor 106 and associated with the assets and/or the THU.
  • the shelving and/or floor space adjacent to the THU may include a license plate or other identifiers that may be detected within the sensor data 104 and usable by the asset management system 102 to recognize and classify the assets and/or the THU.
  • the asset management system 102 may locate and track identifiers on the assets and/or the THU as the assets and/or the THUs are moved about the origination facility, loaded/unloaded from a transport, and/or moved about the destination facility.
  • the THU and/or individual packages may include a bar code or other identifiers that may be detected in and/or extracted from the sensor data 104.
  • the identifiers may be electric, in the form of, for example, an RFID tag, Bluetooth® low energy (BLE) signal, or other wireless communication technology.
  • the asset management system 102 may determine if the asset and/or THU is the assets and/or THUs being tracked or position being estimated, and if so to update the position data of the asset and/or THU accordingly. [0036] Additionally, upon detection, the asset management system 102 may analyze the sensor data 104 to determine if the assets are mislabeled, missing and/or damaged. If the assets are mislabeled, missing or damaged, the asset management system 102 may generate an alert 108 to notify the responsible party 110 associated with the asset (e.g., a facility operator, vehicle operator, transport agent, and the like) as well a third-party system 112 (e.g., an insurance agent, buyer, or other like).
  • the responsible party 110 associated with the asset e.g., a facility operator, vehicle operator, transport agent, and the like
  • a third-party system 112 e.g., an insurance agent, buyer, or other like.
  • the system 102 may determine a pilferage or unauthorized access to the asset and/or THU is underway. In these cases, the asset management system 102 may also send an alert 114 to a law enforcement system 116.
  • the alert 114 may include GPS or global position data associated with the asset and/or THU as well as any identification data usable to track the perpetrator.
  • the asset management system 102 may also generate reports 118 associated with the assets and/or the THUs as the assets are moved at a facility and/or transported between facilities.
  • the reports 102 may include current position, currently responsible party, status (such as damage), last accessed, size, volume, associated identifiers, and the like of the assets.
  • the sensor data 104, alerts 108 and 114, as well as the reports 118 may be sent and/or received by the asset management system 102 via various networks, such as networks 120-126.
  • FIGS. 2-7 are flow diagrams illustrating example processes associated with the asset management system discussed herein.
  • the processes are illustrated as a collection of blocks in a logical flow diagram, which represent a sequence of operations, some or all of which can be implemented in hardware, software, or a combination thereof.
  • the blocks represent computer-executable instructions stored on one or more computer- readable media that, when executed by one or more processor(s), performs the recited operations.
  • computer-executable instructions include routines, programs, objects, components, encryption, deciphering, compressing, recording, data structures and the like that perform particular functions or implement particular abstract data types.
  • FIG. 2 is a flow diagram illustrating an example process 200 associated with determining a change in custody of an asset according to some implementations.
  • the asset management system may determine when custody of an asset has transitioned from one entity to another, such as when an asset is loaded from an originating facility onto a transport.
  • the asset management system may receive first sensor data associated with an asset from a facility or system associated with the facility.
  • the sensor data may be associated with a storage area of a facility, facility gate, and/or a loading or staging area.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of the asset.
  • the sensor data may be analyzed, segmented, classified and the like to determine the presence of an asset within the sensor data.
  • An identity of the segmented and classified data may then be determined based at least in part on an identifier represented with respect to the sensor data and/or based on the characteristics of the segmented and classified objects.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of a facility operator loading a transport with the asset.
  • the first sensor data may include image data of a vehicle, the operator, and/or identifiers associated with the vehicle and/or operator that is loading the transport or transport unit.
  • the segmented and classified sensor data may be used to determine the identity of the operator or a vehicle being operated by the operator.
  • the asset management system may determine, based at least in part on the first sensor data, receipt of the asset at the transport. For example, the asset management system may detect, based at least in part on the first sensor data, a loading event associated with the asset as the asset is moved from the facility to the transport (e.g., the asset exits the facility gate).
  • a loading event associated with the asset as the asset is moved from the facility to the transport (e.g., the asset exits the facility gate).
  • the asset management system may receive second sensor data from the transport.
  • sensors associated with the transport may identify the asset, a THU associated with the asset, or the loading facility operator/vehicle as the facility operator loads the asset into the transport.
  • the asset management system may confirm, based at least in part on the second sensor data, the identity of the asset on the transport.
  • the second sensor data may represent the asset, a THU associated with the asset, an individual or vehicle loading the asset onto the transport or the like. For instance, if the asset is known to be located on a THU that is being moved by a vehicle, such as a forklift, the second sensor data may be processed to determine the identity of the asset, the THU, and/or the forklift.
  • the asset management system may determine, based at least in part on location data received from the transport, a departure of the transport from the facility. For example, the asset management system may receive GPS data from the transport vehicle indicating that the transport is in motion.
  • the asset management system may update a chain of custody associated with the asset based at least in part on the departure and the confirmation of the asset. For example, based on the first sensor data and the second sensor data indicating the asset has been loaded on the transport and the location data indicating the transport has departed the facility, the asset management system may update the chain of custody to reflect that the asset is in the possession and/or care of the transportation company.
  • FIG. 3 is a flow diagram illustrating an example process 300 associated with determining a change in custody of an asset according to some implementations.
  • the asset management system may determine when custody of an asset has transitioned from one entity to another, such as when an asset is unloaded from a transport to an originating facility.
  • the asset management system may receive first sensor data and position data associated with assets from a transport.
  • the sensor data may be associated with a cargo area or hold and/or an opening of the cargo area of a transport.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of the asset.
  • the sensor data may be analyzed, segmented, classified, recognized and the like to determine the presence of an asset within the sensor data.
  • An identity of the segmented and classified data may then be determined based at least in part on an identifier represented with respect to the sensor data and/or based on the characteristics of the segmented and classified objects.
  • the asset management system may detect, based at last in part on the first sensor data, an unloading event associated with the assets.
  • the sensor data may represent the assets being removed from the cargo area via the opening.
  • the asset management system may determine, based at least in part on the first sensor data, receipt of the asset at the facility. For example, the asset management system may detect, based at last in part on the first sensor data, an unloading event associated with the asset as the asset is moved from the transport to the receiving facility (e.g., the asset exits the facility gate). In some cases, the asset management system may segment and classify image data of the unloading event ot detect the asset, a THU that is associated with the asset, an individual moving or caring the asset or the associated THU, a vehicle moving or carrying the asset or the associated THU, and the like.
  • the asset management system may receive second sensor data associated with the asset from a sensor system associated with the receiving facility.
  • sensors associated with a receiving facility may identify the asset, a THU associated with the asset, or the unloading facility operator/vehicle as the facility operator unloads the asset from the transport.
  • the asset management system may confirm, based at least in part on the second sensor data, the identity of the asset.
  • the second sensor data may represent the asset, a THU associated with the asset, an individual or vehicle loading the asset onto the transport or the like. For instance, if the asset is known to be located on a THU that is being moved by a vehicle, such as a forklift, the second sensor data may be processed to determine the identity of the asset, the THU, and/or the forklift.
  • the asset management system may update a chain of custody associated with the asset based at least in part on the confirmation of the asset at the receiving facility. For example, based on the first sensor data and the second sensor data indicating the asset has been unloaded from the transport and accepted by the receiving facility, the asset management system may update the chain of custody to reflect that the asset is in the possession and/or care of the receiving facility.
  • FIG. 4 is a flow diagram illustrating an example process 400 associated with determining damage to an asset according to some implementations. As discussed above, the asset management system may determine when damage has occurred or been detected with respect to an asset. The asset management system may record damage data associated with the asset such that at the time the asset is inspected the value or cost associated with the damage may be applied to the party that caused the damage.
  • the asset management system may receive first sensor data associated with an asset from a facility or system associated with the facility.
  • the sensor data may be associated with a storage area of a facility, facility gate, and/or a loading or staging area.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of the asset.
  • the sensor data may be analyzed, segmented, classified and the like to determine the presence of an asset within the sensor data.
  • An identity of the segmented and classified data may then be determined based at least in part on an identifier represented with respect to the sensor data and/or based on the characteristics of the segmented and classified objects.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of a facility operator loading a transport unit with the asset. For example, the sensor data may again be analyzed, segmented, classified and the like to determine the presence of an operator or a vehicle associated with the operator within the sensor data. An identity of operator may then be determined using the segmented and classified data, such as via feature recognition, identifiers, and the like.
  • the asset management system may determine, based at least in part on the first sensor data, damage data associated with the asset.
  • the damage may be present when the operator picks up or loads the asset from a shelving or storage area and/or caused by the operator during loading.
  • the asset management system may determine the damage using one or more machine learned models and the first sensor data.
  • the asset management system may send an alert to at least the facility operator to notify the facility operator of the damage data, for example, the asset management system may notify the manager at the facility, the operator loading the transport to cause a manual inspection of the asset prior to loading.
  • the system may also recommend alternative assets to load on the transport such that damaged product is not delivered to the receiving party.
  • the asset management system may also alert or notify an owner of the asset or an insurance carrier associated with the asset that damage to the asset has occurred and the estimated extent or costs.
  • the asset management system may update the chain of custody associated with the asset to record the damage data and a location associated with the damage data.
  • the location may be facility and/or a location within the facility.
  • the damage data may include images of the damage, any user inputs resulting from a manual inspection, if an incident occurred (e.g., a forklift impacting the asset, a dropping of the asset, or other incident involving the asset), any data associated with the incident, the identity of the operator and/or vehicle handling the asset at the time the damage was detected, and the like.
  • updating the chain of custody associated with the asset may include updating a block chain record or the like.
  • FIG. 5 is another flow diagram illustrating an example process 500 associated with determining damage to an asset according to some implementations.
  • the asset management system may determine when damage has occurred or been detected with respect to an asset.
  • the asset management system may record damage data associated with the asset such that at the time the asset is inspected the value or cost associated with the damage may be applied to the party that caused the damage.
  • the asset management system may receive first sensor data and location data associated with an asset stored on a transport.
  • the sensor data may be associated with a storage or cargo area of the transport, a gate to the cargo area, and/or the like.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of the asset. For example, the sensor data may be analyzed, segmented, classified and the like to determine the presence of an asset within the sensor data. An identity of the segmented and classified data may then be determined based at least in part on an identifier represented with respect to the sensor data and/or based on the characteristics of the segmented and classified objects or text or barcode/qrcode. [0067] At 506, the asset management system may determine, based at least in part on the first sensor data, damage data associated with the asset. For example, the damage may have occurred during transit, such as by the contents of the cargo area shifting. In some cases, the asset management system may determine the damage using one or more machine learned models and the first sensor data.
  • the asset management system may determine, based at least in part on a chain of custody associated with the asset and the first sensor data that the damage data represents new damage. For example, as discussed above with respect to FIG. 4, the damage may be present at the time the asset was loaded onto the transport. In these cases, the asset management system may confirm that the damage was either present at the time of the loading or new damage occurring during transit. In some cases, the asset management system may utilize one or more machine learned models to compare damage data and determine if the damage is new.
  • the asset management system may send an alert to at least one party associated with the asset to notify the party of the damage data.
  • the asset management system may notify the operator of the transport, a party responsible for the transport (e.g., a manager of the transportation company), an owner of the asset, an insurance carrier associated with the asset, and/or the like.
  • the asset management system may update the chain of custody associated with the asset to record the damage data and the location data associated with the damage data.
  • the location may be a facility and/or a location within the facility.
  • the damage data may include images of the damage, any user inputs resulting from a manual inspection, if an incident occurred (e.g., a traffic accident or the like), any data associated with the incident, the identity of the transport operator, an extent or estimated value of the damage, and/or the like.
  • FIG. 6 is another flow diagram illustrating an example process 600 associated with determining damage to an asset according to some implementations.
  • the asset management system may determine when damage has occurred or been detected with respect to an asset.
  • the asset management system may record damage data associated with the asset such that at the time the asset is inspected the value or cost associated with the damage may be applied to the party that caused the damage.
  • the asset management system may receive first sensor data associated with an asset from a facility or system associated with the facility.
  • the sensor data may be associated with a storage or cargo area of the transport, a gate to the cargo area, and/or the like.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of the asset.
  • the sensor data may be analyzed, segmented, classified and the like to determine the presence of an asset within the sensor data.
  • An identity of the segmented and classified data may then be determined based at least in part on an identifier represented with respect to the sensor data and/or based on the characteristics of the segmented and classified objects.
  • the asset management system may determine, based at least in part on the first sensor data, an identity of a facility operator unloading the asset.
  • the sensor data may again be analyzed, segmented, classified and the like to determine the presence of an operator or a vehicle associated with the operator within the sensor data.
  • An identity of the operator may then be determined using the segmented and classified data, such as via feature recognition, identifiers, and the like.
  • the asset management system may receive second sensor data associated with the asset from the receiving facility.
  • the sensor data may be associated with a storage area of a facility, facility gate, and/or a loading or staging area.
  • the asset management system may confirm, based at least in part on the second sensor data, the identity of the asset.
  • the second sensor data may represent the asset, a THU associated with the asset, an individual or vehicle loading the asset onto the transport or the like. For instance, if the asset is known to be located on a THU that is being moved by a vehicle, such as a forklift, the second sensor data may be processed to determine the identity of the asset, the THU, and/or the forklift.
  • the asset management system may determine, based at least in part on the first sensor data and the second sensor data, damage data associated with the asset.
  • the damage may be present when the operator picks up the asset and/or caused by the operator during unloading.
  • the asset management system may determine the damage using one or more machine learned models, the first sensor data, and the second sensor data.
  • the asset management system may send an alert to at least one party associated with the asset to notify the party of the damage data.
  • the asset management system may notify the operator of the transport, a party responsible for the transport (e.g., a manager of the transportation company), an owner of the asset, an insurance carrier associated with the asset, and/or the like.
  • the asset management system may update the chain of custody associated with the asset to record the damage data and a location associated with the damage data.
  • the location may be a facility, the transport, a location within the facility, and/or a global or GPS position associated with the damage.
  • the damage data may include images of the damage, any user inputs resulting from a manual inspection, if an incident occurred (e.g., a forklift impacting the asset, a dropping of the asset, or other incident involving the asset), any data associated with the incident, the identity of the operator and/or vehicle handling the asset at the time the damage was detected, and the like.
  • FIG. 7 is a flow diagram illustrating an example process 700 associated with providing asset security according to some implementations.
  • one or more assets may be missing upon a transport’s arrival at a receiving facility.
  • assessing and assigning liability due to the missing assets may be difficult.
  • the originating facility may have failed to load the asset onto the transport and/or the asset may have been accessed and removed during transit.
  • the asset may even be misplaced during unloading by the receiving facility.
  • the process 700 discusses on process for determining if the assets were accessed during transit.
  • the asset management system may receive location data associated with a transport currently in possession of an asset.
  • the location data may be global position data such as GPS data associated with a GPS sensor of the transport and/or the asset.
  • the asset management system may determine, based on the location data, that the transport has stopped and a current location of the transport. In some cases, the asset management system may determine that the transport has stopped for greater than a threshold period of time (such as a time typically associated with a traffic light or other indicator). In these cases, the system may activate sensors within a hold, storage area, or cargo area of the transport to monitor the status of the assets.
  • a threshold period of time such as a time typically associated with a traffic light or other indicator.
  • the asset management system may determine, based at least in part on sensor data associated with an interior of the transport, that an individual is accessing assets stored in the transport.
  • the sensor data may represent an opening of a gate associated with the storage area of the transport.
  • the sensor data may detect the presence and/or identity of the individual accessing the hold.
  • the asset management system may determine, based at least in part on the sensor data, an identity of the individual.
  • the individual may be an operator of the transport or another individual associated with the transport company, the assets, or the like.
  • the identity of the individual may be unknown.
  • the asset management system may determine, based at least in part on the sensor data, a pilferage is in process.
  • the system may determine the identity of the individual accessing the hold, a current location is not a delivery location, and/or that a package is being tampered with.
  • the individual accessing the hold may be unauthorized.
  • an individual may be authorized individual to access the storage area but not the assets.
  • the system may determine the pilferage is in process upon the individual opening or removing assets from the hold. Accordingly, in some cases, the asset management system may determine the pilferage based on the sensor data representing an access of a THU or the assets themselves.
  • the asset management system may send a first alert associated with the pilferage to a law enforcement agency.
  • the system may determine the local law enforcement authority and send the alert.
  • the first alert including data associated with the individual accessing the asset, a location of the transport, other data associated with the transport (such as identification data), data associated with the accessed assets, and the like.
  • the asset management system may send a second alert to a party associated with the transport.
  • the system may send the alert to a manager of the transport to notify the manger as to the pilferage.
  • the second alert may also include data associated with the individual accessing the asset, a location of the transport, other data associated with the transport (such as identification data), data associated with the accessed assets, and the like.
  • the asset management system may send a third alert to a party associated with the assets.
  • the system may send the alert to a party associated with the originating facility, the receiving facility, an insurance provider, and the like.
  • the third alert may include data associated with the individual accessing the asset, a location of the transport, other data associated with the transport (such as identification data), data associated with the accessed assets, and the like.
  • the asset management system may update the chain of custody associated with the asset to record pilferage data associated with the pilferage.
  • the pilferage data may also include data associated with the individual accessing the asset, a location of the transport, other data associated with the transport (such as identification data), data associated with the accessed assets, and the like.
  • the asset management system may also determine who is responsible for paying for or the costs associated with the pilferage. For instance, the asset management system may determine who is responsible based at least in part on the chain of custody and the sensor data.
  • FIG. 8 is an example sensor system 800 that may implement the techniques described herein according to some implementations.
  • the sensor system 800 may include one or more communication interfaces(s) 802 that enable communication between the system 800 and one or more other local or remote computing device(s) or remote services, such as an asset management system of FIGS. 1-7.
  • the communication interface(s) 802 can facilitate communication with other proximity sensor systems, a central control system, or other facility systems.
  • the communications interfaces(s) 802 may enable Wi-Fi-based communication such as via frequencies defined by the IEEE 802.11 standards, short range wireless frequencies such as Bluetooth, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communication, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables the respective computing device to interface with the other computing device(s).
  • Wi-Fi-based communication such as via frequencies defined by the IEEE 802.11 standards, short range wireless frequencies such as Bluetooth, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communication, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables the respective computing device to interface with the other computing device(s).
  • the one or more sensor(s) 804 may be configured to capture the sensor data 826 associated with assets.
  • the sensor(s) 804 may include thermal sensors, time-of-flight sensors, location sensors, LIDAR sensors, SWIR sensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g., RGB, IR, intensity, depth, etc.), Muon sensors, microphone sensors, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), 5G or other wireless sensors, and the like.
  • the sensor(s) 804 may include multiple instances of each type of sensor. For instance, camera sensors may include multiple cameras disposed at various locations.
  • the sensor system 800 may also include one or more location determining component(s) 806 for determining a global position of the assets, a transport associated with the assets, a THU associated with the assets, or the like.
  • the location determining component(s) 806 may include one or more sensor package combinations including Global Navigation Satellite System (GNSS) sensors and receivers, Global Positioning System (GPS) sensors and receivers, or other satellite systems.
  • GNSS Global Navigation Satellite System
  • GPS Global Positioning System
  • the location determining component(s) 806 may be configured to decode satellite signals in various formats or standards, such as GPS, GLONASS, Galileo, BeiDou.
  • the location determining component(s) 806 may be placed at various places associated with the assets, THU, and/or transports to improve the accuracy of the coordinates determined form the data received by each of the location determining component(s) 806.
  • the sensor system 800 may include one or more processors 808 and one or more computer-readable media 810. Each of the processors 808 may itself comprise one or more processors or processing cores.
  • the computer-readable media 810 is illustrated as including memory/storage.
  • the computer-readable media 810 may include volatile media (such as random access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth).
  • the computer-readable media 810 may include fixed media (e.g., GPU, NPU, RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., Flash memory, a removable hard drive, an optical disc, and so forth).
  • the computer-readable media 810 may be configured in a variety of other ways as further described below.
  • modules such as instructions, data stores, and so forth may be stored within the computer-readable media 810 and configured to execute on the processors 808.
  • the computer-readable media 810 stores data capture instructions 812, data extraction instructions 814, identification instructions 816, damage inspection instructions 818, event determining instructions 820, pilferage detection instructions 822, alert instructions 824, as well as other instructions, such as an operating system.
  • the computer-readable media 810 may also be configured to store data, such as sensor data 826 and machine learned models 828, and location data 830 as well as other data.
  • the data capture instructions 812 may be configured to utilize or activate the sensor systems 804 to capture sensor data 826 associated with an asset, a THU, a region of the facility or transport, or the like.
  • the captured sensor data 626 may then be stored and/or transmitted or streamed to an asset management system, as discussed herein.
  • the data extraction instructions 814 may be configured to extract, segment, classify objects represented within the sensor data 826.
  • the data extraction instructions 814 may segment and classify each asset present on a THU as well as other objects or features within the sensor data 826, such as individuals and/or vehicles.
  • the data extraction instructions 814 may utilize the machine learned models 828 to perform extraction, segmentation, classification, recognition, and the like.
  • the identification instructions 816 may be configured to determine an identity of an asset, a THU, a region of the facility or transport, or an individual and/or vehicle, and the like.
  • the identification instructions 816 may utilize one or more machine learned models 828 with respect to the sensor data 826 and/or the extracted data to determine the identity of an asset, a THU, a location, and region of the facility or transport, or an individual and/or vehicle, and the like, as discussed above.
  • the damage inspection instructions 818 may be configured to process the sensor data 826 to identify damage associated with an asset and/or a THU. For example, the damage inspection instructions 818 may detect damage using the machine learned models then compare the damage detected with any known damage to determine if the damage was received while the THU was being moved. In some cases, the damage inspection instructions 818 may also rate the damage, for instance, using a severity rating and/or value the damage (such as for insurance purposes).
  • the event determining instructions 820 may be configured to process the sensor data 826 to determine if a loading or unloading (e.g., pickup or delivery) event is in process and to cause the processors 808 to perform various operations based on the determination of the event type.
  • a loading or unloading (e.g., pickup or delivery) event is in process and to cause the processors 808 to perform various operations based on the determination of the event type.
  • the pilferage detection instructions 822 may be configured to process the sensor data 826 to identify a pilferage associated with an asset and/or a THU. For example, the pilferage detection instructions 822 may detect the pilferage using the machine learned models 828, the sensor data 826, and/or the location data 830. In some cases, the pilferage detection instructions 822 may identify the individual(s) responsible for the pilferage.
  • the alert instructions 824 may be configured to alert or otherwise notify vehicle operators, facility operators, managers, insurance carriers, government agencies, and the like in response to detection of damage, pilferage, loading, unloading, and the like with respect to one or more assets.
  • FIG. 9 is an example asset management system 900 that may implement the techniques described herein according to some implementations.
  • the asset management system 900 may include one or more communication interface(s) 902 (also referred to as communication devices and/or modems).
  • the one or more communication interfaces(s) 902 may enable communication between the system 900 and one or more other local or remote computing device(s) or remote services, such as sensors system of FIG. 8.
  • the communication interface(s) 902 can facilitate communication with other proximity sensor systems, a central control system, or other facility systems.
  • the communications interfaces(s) 902 may enable Wi-Fi-based communication such as via frequencies defined by the IEEE 802.11 standards, short range wireless frequencies such as Bluetooth, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communication, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables the respective computing device to interface with the other computing device(s).
  • the asset management system 900 may include one or more processors 908 and one or more computer-readable media 910. Each of the processors 908 may itself comprise one or more processors or processing cores.
  • the computer-readable media 910 is illustrated as including memory/storage.
  • the computer-readable media 910 may include volatile media (such as random access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth).
  • volatile media such as random access memory (RAM)
  • nonvolatile media such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth.
  • the computer-readable media 910 may include fixed media (e.g., GPU, NPU, RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., Flash memory, a removable hard drive, an optical disc, and so forth).
  • the computer- readable media 910 may be configured in a variety of other ways as further described below.
  • the computer-readable media 910 stores data extraction instructions 914, identification instructions 916, damage determining instructions 918, event determining instructions 920, pilferage detection instructions 922, alert instructions 924, as well as other instructions, such as an operating system.
  • the computer-readable media 910 may also be configured to store data, such as sensor data 926, machine learned models 928, and location data 930 as well as other data.
  • the data extraction instructions 914 may be configured to extract, segment, classify objects represented within the sensor data 926.
  • the data extraction instructions 914 may segment and classify each asset present on a THU as well as other objects or features within the sensor data 926, such as individuals and/or vehicles.
  • the data extraction instructions 914 may utilize the machine learned models 928 to perform extraction, segmentation, classification, recognition, and the like.
  • the identification instructions 916 may be configured to determine an identity of an asset, a THU, region of the facility or transport, and individual and/or vehicle, and the like.
  • the identification instructions 916 may utilize one or more machine learned models 928 with respect to the sensor data 926 and/or the extracted data to determine the identity of an asset, a THU, a location, and region of the facility or transport, and individual and/or vehicle, and the like, as discussed above.
  • the damage inspection instructions 918 may be configured to process the sensor data 926 to identify damage associated with an asset and/or a THU. For example, the damage inspection instructions 918 may detect damage using the machine learned models then compare the damage detected with any known damage to determine if the damage was received while the THU was being moved. In some cases, the damage inspection instructions 918 may also rate the damage, for instance, using a severity rating and/or value the damage (such as for insurance purposes).
  • the event determining instructions 920 may be configured to process the sensor data 926 to determine if a loading or unloading (e.g., pickup or delivery) event is in process and to cause the processors 908 to perform various operations based on the determination of the event type.
  • a loading or unloading (e.g., pickup or delivery) event is in process and to cause the processors 908 to perform various operations based on the determination of the event type.
  • the pilferage detection instructions 922 may be configured to process the sensor data 926 to identify a pilferage associated with an asset and/or a THU. For example, the pilferage detection instructions 922 may detect the pilferage using the machine learned models 928, the sensor data 926, and/or the location data 930. In some cases, the pilferage detection instructions 922 may identify the individual(s) responsible for the pilferage.
  • the alert instructions 924 may be configured to alert or otherwise notify vehicle operators, facility operators, managers, insurance carriers, government agencies, and the like in response to detection of damage, pilferage, loading, unloading, and the like with respect to one or more assets.
  • FIG. 10 is an example pictorial view 1000 associated with the systems of FIGS. 1-9 according to some implementations.
  • an operator 1002 is loading a transport 1004 with an asset 1006.
  • sensors generally indicated by 1008, may capture sensor data 1010 associated with the loading event and provide to the asset management system 1012 the sensor data 1010, to update the chain of custody associated with the assets 1006 as discussed here.
  • FIG. 11 is another example pictorial view associated with the systems of FIGS. 1-9 according to some implementations.
  • an operator 1102 is unloading an asset 1106 from a transport 1104.
  • sensors generally indicated by 1108, may capture sensor data 1110 associated with the unloading event and provide to the asset management system 1112 the sensor data 1110, to update the chain of custody associated with the assets 1106 as discussed here.
  • FIG. 12 is another example pictorial view associated with the systems of FIGS. 1-9 according to some implementations.
  • an operator 1202 is loading a transport 1204 with an asset 1206.
  • sensors generally indicated by 1208, may capture sensor data 1210 associated with the loading event and provide to the asset management system 1212 the sensor data 1210, to update the chain of custody associated with the assets 1206 as discussed here.
  • a system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving first sensor data associated with a first physical environment; determining, based at least in part on the first sensor data, an identity of an asset; determining, based at least in part on the first sensor data, a type of event associated with the asset; receiving second sensor data associated with a second physical environment; determining, based at least in part on the second sensor data, the identity of the asset; updating a chain of custody associated with the asset responsive to determining the identity of the asset based at least in part on the second sensor data.
  • determining the identity of the asset comprise at least one of determining the identity of at least one of: a transport handling unit; the asset; an individual associated with the asset; or a vehicle associated with the asset.

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Quality & Reliability (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Development Economics (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

L'invention concerne un système de surveillance et de suivi d'actif stocké dans une installation et pendant le transit entre des installations. Par exemple, un système de gestion d'actif peut être configuré pour suivre un actif lorsque l'actif est chargé au niveau d'une installation d'origine, tandis que l'actif est en transit, et lors de l'arrivée au niveau d'une installation de destination. Le système de gestion d'actif peut maintenir une chaîne de traçabilité associée à l'actif.
PCT/US2022/075377 2021-08-24 2022-08-24 Système de suivi d'actif WO2023028507A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163236290P 2021-08-24 2021-08-24
US63/236,290 2021-08-24

Publications (1)

Publication Number Publication Date
WO2023028507A1 true WO2023028507A1 (fr) 2023-03-02

Family

ID=85322210

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/075377 WO2023028507A1 (fr) 2021-08-24 2022-08-24 Système de suivi d'actif

Country Status (1)

Country Link
WO (1) WO2023028507A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050092823A1 (en) * 2003-10-30 2005-05-05 Peter Lupoli Method and system for storing, retrieving, and managing data for tags
US20070021100A1 (en) * 2001-08-17 2007-01-25 Longview Advantage, Inc. System for asset tracking
US20070164858A1 (en) * 2003-06-17 2007-07-19 Intelagents, Inc. Global intelligent remote detection system
US20080040244A1 (en) * 2006-08-08 2008-02-14 Logcon Spec Ops, Inc. Tracking and Managing Assets

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070021100A1 (en) * 2001-08-17 2007-01-25 Longview Advantage, Inc. System for asset tracking
US20070164858A1 (en) * 2003-06-17 2007-07-19 Intelagents, Inc. Global intelligent remote detection system
US20050092823A1 (en) * 2003-10-30 2005-05-05 Peter Lupoli Method and system for storing, retrieving, and managing data for tags
US20080040244A1 (en) * 2006-08-08 2008-02-14 Logcon Spec Ops, Inc. Tracking and Managing Assets

Similar Documents

Publication Publication Date Title
US11526973B2 (en) Predictive parcel damage identification, analysis, and mitigation
US20230161351A1 (en) System for monitoring inventory of a warehouse or yard
US11249169B2 (en) Site matching for asset tracking
US20210271704A1 (en) System and Method for Identifying Objects in a Composite Object
US20230114688A1 (en) Edge computing device and system for vehicle, container, railcar, trailer, and driver verification
Whig et al. Role of AI and IoT in Intelligent Transportation
JP2024520533A (ja) 在庫追跡のためのシステム
WO2023028507A1 (fr) Système de suivi d'actif
US20230410029A1 (en) Warehouse system for asset tracking and load scheduling
WO2023028509A2 (fr) Système pour déterminer des opérations d'entretien et de réparation
US20240078499A1 (en) System for monitoring transportation, logistics, and distribution facilities
WO2024044174A1 (fr) Système et procédé de chargement d'un conteneur
WO2024030563A1 (fr) Système de contrôle d'entrée et de sortie de parc
WO2023172953A2 (fr) Système et procédés pour effectuer des audits de panier de commande
US20240071046A1 (en) System and Method for Load Bay State Detection
US20230098677A1 (en) Freight Management Systems And Methods
US20230101794A1 (en) Freight Management Systems And Methods
Neetha et al. Real-Time Motion Detection for Cargo Tracking and Management in Industrial Warehouses
WO2023055714A1 (fr) Systèmes et procédés de gestion de fret
Bierwirth et al. SmartAirCargoTrailer: Autonomous short distance transports in air cargo

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22862243

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE