JP7513366B2 - Shipment and receipt management system - Google Patents

Description

The present invention relates to an inbound/outbound management system that determines the behavior of cargo transport means.

In general, the storage location of an item in a warehouse from the time the item arrives at the warehouse until it is shipped is determined by a management device in accordance with a specific program (see, for example, Patent Document 1).

The product placement system of Patent Document 1 has a grouping means that calculates the amount of change in shipping quantity, shipping volume pattern, and shipping volume peak position, and classifies multiple products into multiple product groups based on the magnitude of the amount of change, a shipping prediction means that predicts the shipping quantity of products based on past shipping records, and a placement location determination means that determines the placement location for each classified product group and the placement locations of multiple products based on the shipping prediction of the shipping prediction means.

However, in Patent Document 1, the behavior of the logistics robot (transport means) for efficiently shipping goods is not taken into consideration, and there is a risk of delays in the transport of goods due to, for example, long transport distances or congested transport routes, and there are cases where the goods cannot be transported to the shipping location at the scheduled shipping time.

Patent No. 6457705

The present invention was made in consideration of the above circumstances, and aims to provide an incoming/outgoing management system that can transport packages to the shipping location at the scheduled shipping time by preventing delays in package transport.

In order to solve the above problems, the present invention provides an incoming/outgoing management system that includes an information acquisition unit that acquires luggage information related to luggage stored in a warehouse and transport means information related to a transport means for transporting the luggage, and a behavior decision unit that determines the behavior of the transport means by performing reinforcement learning based on the luggage information and the transport means information , and the information acquisition unit acquires, as the luggage information, first luggage information related to luggage that has arrived at the warehouse and second luggage information related to luggage that is scheduled to arrive at the warehouse, and the first luggage information includes information indicating a location of the luggage and a shipping schedule when the luggage will be shipped. The second luggage information includes information relating to time and information relating to the shipping location from which the luggage will be shipped, the second luggage information includes information relating to the scheduled arrival time when the luggage will arrive and information relating to the arrival location where the luggage will arrive, and the transportation means information includes information relating to the current location of the loading vehicle, and the behavior decision unit determines the behavior of the transportation means that maximizes the reward , assuming that a reward is obtained when the luggage is transported to the shipping location of the luggage at the scheduled shipping time of the luggage, and that the reward is reduced when other luggage is present at the arrival location of the luggage at the scheduled arrival time of the luggage.
In addition, the incoming/outgoing management system of the present invention comprises an information acquisition unit that acquires luggage information related to luggage stored in a warehouse and transportation means information related to the transportation means for transporting the luggage, and a behavior decision unit that determines the behavior of the transportation means by performing reinforcement learning based on the luggage information and the transportation means information, wherein the luggage information includes information indicating the location of the luggage, information related to the scheduled shipping time at which the luggage will be shipped, and information related to the shipping location at which the luggage will be shipped, and the transportation means information includes information related to the current location of the loading vehicle, information related to the orientation of the loading vehicle, and information related to the loading lift height of the loading vehicle , and the behavior decision unit determines the behavior of the loading vehicle that will maximize the reward, assuming that a reward is obtained when the luggage is transported to the shipping location of the luggage at the scheduled shipping time of the luggage.

In addition, it is preferable that the behavior decision unit decides on the behavior of the loading vehicle that will maximize the reward, assuming that the reward is reduced when the cargo is not transported to the shipping location of the cargo at the scheduled shipping time of the cargo.

In addition , it is preferable that the second package information further includes information related to a scheduled shipping time when the package will be shipped, and information related to a shipping location when the package will be shipped.

In addition, it is preferable that the behavior decision unit decides on the behavior of the loading/ unloading vehicle that maximizes the reward, assuming that the reward is reduced when other luggage is present at the arrival location of the luggage at the scheduled arrival time of the luggage.

In addition, it is preferable that the behavior decision unit determines, as the behavior of the loading vehicle , the storage location of the luggage in the warehouse, the timing of transporting the luggage from the receiving location to the storage location, and the timing of transporting the luggage from the storage location to the shipping location.

It is also preferable that the system further comprises a loading/unloading command unit which commands the loading vehicle to carry out the action determined by the action determination unit, and the loading vehicle is an unmanned transport vehicle which transports luggage based on the command of the loading/unloading command unit.

The present invention provides an incoming/outgoing management system that can prevent delays in the transport of packages and transport packages to the shipping location at the scheduled shipping time.

1 is a schematic diagram of an incoming/outgoing management system according to an embodiment of the present invention; FIG. 2 is a plan view showing the configuration inside the warehouse according to the embodiment. 11 is an explanatory diagram showing specific examples of transportation means information, first package information, and second package information. FIG.

A receipt/shipment management system 1 according to an embodiment of the present invention will be described with reference to FIGS.
As shown in FIG. 1, the receipt/shipment management system 1 includes an receipt/shipment management device 10 that manages the receipt and shipment of luggage N, an unmanned transport vehicle 20 that is a means for transporting luggage N, and a rack 30 for storing luggage N.

The incoming/outgoing management device 10 is a device that manages the cargo N received at the warehouse S until the cargo N is shipped from the warehouse S. The incoming/outgoing management device 10 includes an information acquisition unit 11, a memory unit 12, an action decision unit 13, and a loading/unloading command unit 14. The incoming/outgoing management device 10 is configured to be able to communicate with an external terminal T and an unmanned transport vehicle 20.

The information acquisition unit 11 is a device that acquires transport means information and luggage information. Specifically, for example, the information acquisition unit 11 is composed of a communication device that acquires transport means information from the automated guided vehicle 20 and a communication device that acquires luggage information from an external terminal T such as an external server. The transport means information is information related to the automated guided vehicle 20 that operates in the warehouse S. The luggage information is information related to luggage N stored in the warehouse S. The luggage information includes first luggage information related to luggage N that has arrived at the warehouse S and second luggage information related to luggage N that is scheduled to arrive at the warehouse S. The transport means information, first luggage information, and second luggage information will be described later with reference to FIG. 3.

The memory unit 12 is a device that stores the information acquired by the information acquisition unit 11. The transport means information stored in the memory unit 12 is updated as appropriate according to the current status of the unmanned transport vehicle 20. Similarly, the baggage information stored in the memory unit 12 is updated as appropriate according to the current status of the baggage N.

The memory unit 12 also stores information related to the reinforcement learning of the action decision unit 13. Specifically, the memory unit 12 stores, for example, state transition probabilities and reward acquisition probabilities obtained in the process of reinforcement learning described below. The state transition probabilities and reward acquisition probabilities stored in the memory unit 12 are updated as appropriate in the process of reinforcement learning.

The behavior decision unit 13 is a device that determines the behavior of the unmanned carrier 20 by performing reinforcement learning based on the baggage information and the transportation means information. The behavior decision unit 13 determines the behavior of the unmanned carrier 20 that maximizes the reward, which is an evaluation value. Specifically, the behavior decision unit 13 determines the behavior of the unmanned carrier 20 that maximizes the reward, assuming that the reward is obtained when the baggage N is transported to the shipping location of the baggage N at the scheduled shipping time of the baggage N, that the reward is reduced (i.e., a negative reward (penalty) is obtained) when the baggage N is not transported to the shipping location of the baggage N at the scheduled shipping time of the baggage N, and that the reward is reduced when another baggage N exists at the arrival location of the baggage N at the scheduled arrival time of the baggage N. Specifically, for example, the behavior decision unit 13 determines, as the behavior of the automated guided vehicle 20, the storage location or temporary storage location of the luggage N in the warehouse S, the transport timing of the luggage N from the receiving location to the storage location or temporary storage location, the transport timing of the luggage N from the temporary storage location to the storage location, and the transport timing of the luggage N from the storage location to the shipping location.

The loading and unloading command unit 14 is a device that commands the unmanned transport vehicle 20 to carry out the action determined by the action determination unit 13. The loading and unloading command unit 14 issues loading and unloading commands directly or indirectly to the unmanned transport vehicle 20.

The unmanned transport vehicle 20 is a mobile body capable of autonomous movement and transporting luggage N, and transports luggage N based on commands from the loading and unloading command unit 14. In this embodiment, the unmanned transport vehicle 20 is an unmanned forklift equipped with forks 21 for lifting and lowering luggage N.

The rack 30 is a stand provided in a storage location for the luggage N. The rack 30 is equipped with shelves 31 for supporting the luggage N.

An example of the environment of a warehouse S in which the automated guided vehicle 20 handles cargo will be described with reference to FIG.
As shown in FIG. 2, a warehouse S is provided with, for example, receiving locations R1 to R3, storage locations K1 to K60, temporary storage locations T1 to T3, and shipping locations F1 to F3.

The receiving locations R1 to R3 are locations where luggage N that arrives at the warehouse S is placed. The luggage N placed at the receiving locations R1 to R3 is transported by the automated guided vehicle 20 to an available storage location K1 to K60 or a temporary storage location T1 to T3.

Storage locations K1 to K60 are locations within warehouse S where luggage N is stored. When luggage N placed in storage locations K1 to K60 is shipped, it is transported by automated guided vehicle 20 to shipping locations F1 to F3. Racks 30 are provided in storage locations K1 to K60, and each of storage locations K1 to K60 includes storage locations S1 to S3 (see FIG. 1) with multiple tiers separated by shelves 31 of rack 30.

The temporary storage locations T1 to T3 are locations where luggage N is temporarily stored. The luggage N placed in the temporary storage locations T1 to T3 is transported by the automated guided vehicle 20 to vacant storage locations K1 to K60. In order to eliminate congestion at the receiving locations R1 to R3, it is preferable that the temporary storage locations T1 to T3 are located closer to the receiving locations R1 to R3 than the vacant storage locations K1 to K60.

Shipping locations F1 to F3 are locations where packages N to be shipped from warehouse S are placed. Packages N placed at shipping locations F1 to F3 are transported outside warehouse S by transport means (not shown) other than automated guided vehicle 20.

The information for determining the behavior of the unmanned transport vehicle 20 will be described with reference to Figures 3(A) to (C). Figure 3(A) shows an example of transport means information, which is information related to the unmanned transport vehicle 20. Figure 3(B) shows an example of first parcel information, which is information related to parcels N that have arrived at the warehouse S. Figure 3(C) shows an example of second parcel information, which is information related to parcels N that are scheduled to arrive at the warehouse S.

As shown in FIG. 3(A), the transport means information includes, for example, information related to the current status of the unmanned transport machine 20, the current location of the unmanned transport machine 20, the orientation of the unmanned transport machine 20, and the lifting height of the unmanned transport machine 20. The current status of the unmanned transport machine 20 indicates whether the unmanned transport machine 20 is transporting cargo N. The current location of the unmanned transport machine 20 indicates the position of the unmanned transport machine 20 within the warehouse S, the orientation of the unmanned transport machine 20 indicates the attitude angle of the unmanned transport machine 20 relative to a reference direction, and the lifting height of the unmanned transport machine 20 indicates the lifting height of the forks 21 provided on the unmanned transport machine 20.

The current location (position), orientation (attitude angle), and loading height (height of the forks 21) of the unmanned transport vehicle 20 are elements of a finite set of states in a Markov decision process, and represent the state of the unmanned transport vehicle 20 in a discrete manner. The position of the unmanned transport vehicle 20 is represented as a discretized point obtained by dividing multiple routes that the unmanned transport vehicle 20 can travel by a predetermined travel time t. The discretized point that can indicate the position of the unmanned transport vehicle 20 is configured to necessarily include a branch point of multiple routes. Specifically, for example, the dashed dotted lines in FIG. 2 indicate the routes that the unmanned transport vehicle 20 can travel, and the black points connecting the dashed dotted lines are discretized points that can indicate the position of the unmanned transport vehicle 20. The height of the forks 21 is also discretized according to the height of the shelf 31, for example.

As shown in FIG. 3(B), the first parcel information includes, for example, an identifier for identifying each parcel N, the current status of parcel N, the arrival location of parcel N, the temporary storage location of parcel N, the storage location of parcel N, the scheduled shipping time at which parcel N will be shipped, and information related to the shipping location at which parcel N will be shipped. The current status of parcel N indicates the position of parcel N. The position of parcel N is the current location of the unmanned transport vehicle 20 transporting parcel N if the current status of parcel N is being transported, the arrival location of parcel N if the current status of parcel N is not in storage, the temporary storage location of parcel N if the current status of parcel N is being temporarily stored, and the storage location of parcel N if the current status of parcel N is being stored. The scheduled shipping time indicates the remaining time until the scheduled shipping time.

As shown in FIG. 3(C), the second parcel information includes, for example, an identifier for identifying each parcel N, the scheduled arrival time of parcel N, the arrival location where parcel N will arrive, the scheduled shipping time of parcel N, and the shipping location where parcel N will be shipped. The scheduled arrival time indicates the time remaining until the scheduled arrival time, and the scheduled shipping time indicates the time remaining until the scheduled shipping time.

The position of luggage N (i.e., the current location of the unmanned transport vehicle 20 transporting luggage N, the storage location of luggage N, the temporary storage location of luggage), the arrival location of luggage N, the shipping location of luggage N, the scheduled arrival time of luggage N, and the scheduled shipping time of luggage N are each elements of a finite state set of the Markov decision process, and represent the state of luggage N in a discrete manner. The scheduled arrival time and scheduled shipping time of luggage N are each discretized by the travel time t corresponding to the interval between the discretized points that can indicate the position of the unmanned transport vehicle 20.

In the thus configured receiving/shipping management system 1, the behavior decision unit 13, which performs reinforcement learning, observes the current state s (the state indicated by the information acquired by the information acquisition unit 11) in an environment formulated as a Markov decision process, and determines the behavior a (i.e., the behavior of the unmanned transport vehicle 20) that transitions to a new state s'. At this time, a reward corresponding to the transition to state s' is generated, and the state s' and reward after the transition depend on the current state s and the behavior a. The behavior a is, for example, moving (running) to another discretized point via an arbitrary discretized point, changing the attitude angle (turning), changing the lifting height of the loading/unloading device (raising and lowering the fork 21), or a combination of these. The Markov decision process can be solved by a value iteration method or a policy iteration method, and the reinforcement learning algorithm can be, for example, a Q-learning algorithm, which is a reinforcement learning version of the value iteration method.

By determining the action a so as to maximize the reward (i.e. so as to avoid a decrease in the reward), it is expected that the unmanned carrier 20 will take the following actions (A) to (C), for example.
(A) When the frequency of trips between storage locations K1-K30 and shipping locations F1-F3 increases due to a large number of shipments of luggage N, luggage N stored in storage locations K1-K30 that are relatively far from the shipping locations F1-F3 is transported in advance to storage locations K31-K60 that are relatively close to the shipping locations F1-F3 prior to the shipment of luggage N.
(B) When the handling of goods from the receiving locations R1 to R3 to the storage locations K1 to K60 becomes slow due to a large amount of incoming goods N, the goods N are transported from the receiving locations R1 to R3 to the temporary storage locations T1 to T3, and then, when the amount of incoming goods N decreases, the goods N are transported from the temporary storage locations T1 to T3 to the storage locations K1 to K60.
(C) When the handling of cargo from the receiving locations R1 to R3 to the storage locations K31 to K60 becomes slow due to a large amount of cargo N arriving, the cargo N is transported to the storage locations K1 to K30 which are relatively close to the receiving locations R1 to R3, and then, when the amount of cargo N arriving decreases, the cargo N is transported from the storage locations K1 to K30 to the storage locations K31 to K60.

The present embodiment provides the following effects.
(1) The parcel information includes information on the scheduled shipping time when the parcel N is to be shipped and information on the shipping location from which the parcel N is to be shipped, and the behavior decision unit 13, which performs reinforcement learning based on the parcel information and the like, determines the behavior of the unmanned transport vehicle 20 that maximizes the reward, assuming that the reward is obtained when the parcel N is transported to the shipping location of the parcel N at the scheduled shipping time of the parcel N. According to this configuration, the behavior of the unmanned transport vehicle 20 is determined by the reinforcement learning of the behavior decision unit 13, so that it is possible to save the effort of creating a program for efficiently shipping goods according to the environment, and by the unmanned transport vehicle 20 taking the behavior determined by the behavior decision unit 13, it is possible to prevent delays in transporting the parcel N to the shipping locations F1 to F3, and the parcel N can be transported to the shipping locations F1 to F3 at the scheduled shipping time.

(2) The behavior decision unit 13 determines the behavior of the automated guided vehicle 20 that maximizes the reward, assuming that the reward is reduced when the package N is not delivered to the shipping location of the package N at the scheduled shipping time of the package N. With this configuration, it is possible to further prevent delays in the delivery of the package N to the shipping locations F1 to F3.

(3) The information acquisition unit 11 acquires first package information related to package N that has arrived at the warehouse S and second package information related to package N that is scheduled to arrive at the warehouse S. With this configuration, the behavior decision unit 13 can decide the behavior of the automated guided vehicle 20 based on the information related to package N that is scheduled to arrive at the warehouse S, for example, to speed up the handling of packages at the receiving locations R1 to R3.

(4) The behavior decision unit 13 determines the behavior of the unmanned transport vehicle 20 that maximizes the reward, assuming that the reward is reduced when another package N is present at the receiving location of the package N at the scheduled arrival time of the package N. That is, when two packages N have the same receiving location but different scheduled arrival times, the package A has the earlier scheduled arrival time and the package B has the later scheduled arrival time, the behavior decision unit 13 determines the behavior of the unmanned transport vehicle 20 that maximizes the reward, assuming that the reward is reduced when package A has not been transported from the receiving location to another location at the scheduled arrival time of package B. With this configuration, for example, delays in transporting package N from receiving locations R1 to R3 to storage locations K1 to K60 or temporary storage locations T1 to T3 can be prevented.

(5) The behavior decision unit 13 decides, as the behavior of the automated guided vehicle 20, the storage location of the luggage N in the warehouse S, the timing of transporting the luggage N from the receiving location to the storage location, and the timing of transporting the luggage N from the storage location to the shipping location. With this configuration, it is not necessary to create a program that decides the storage location of the luggage N in the warehouse S, etc.

(6) The incoming/outgoing management system 1 includes a loading/unloading command unit 14 that commands the unmanned transport vehicle 20 to take the action determined by the action determination unit 13. According to this configuration, the baggage N is automatically transported by the unmanned transport vehicle 20 that operates based on the command of the loading/unloading command unit 14.

The present invention is not limited to the above embodiment, and the above configuration can be modified. For example, the following modifications can be implemented, or the following modifications can be combined to implement the present invention.

The receipt-shipment management system 1 may include a plurality of unmanned transport vehicles 20. In this case, the transport means information includes information related to an identifier for identifying each individual unmanned transport vehicle 20, and at least one of the baggage information and the transport means information includes information that associates a baggage N with the unmanned transport vehicle 20 that transports the baggage N. In such a configuration, for example, the unmanned transport vehicle 20 is expected to take the following action (D).
(D) When the receiving locations R1 to R3 become congested due to a large amount of incoming luggage N, the luggage N is transported from the receiving locations R1 to R3 to the temporary storage locations T1 to T3, and then, when the amount of incoming luggage N decreases, the luggage N is transported from the temporary storage locations T1 to T3 to the storage locations K1 to K60.

- The unmanned transport vehicle 20 may be a loading vehicle other than an unmanned forklift. Furthermore, the transport means is not limited to the unmanned transport vehicle 20, but may be a worker who takes an action determined by the action decision unit 13. In this case, the worker as the transport means refers to a device (e.g., a display device that outputs an image) that presents the action determined by the action decision unit 13, and transports the luggage N according to the presentation.

- Information related to state s (i.e., information related to the unmanned conveying machine 20 and information related to the luggage N) is not limited to the information listed in the above embodiment, and may include other information. In other words, the elements of the state set related to the unmanned conveying machine 20 and the elements of the state set related to the luggage N are not limited to only the elements described in the above embodiment. For example, if the unmanned conveying machine 20 is a forklift equipped with a reach mechanism that moves the fork 21 forward and backward, it is preferable that the conveying means information includes information related to the forward and backward positions of the fork 21, if the unmanned conveying machine 20 is a forklift equipped with a tilt mechanism that tilts the fork 21 forward and backward, it is preferable that the conveying means information includes information related to the tilt angle of the fork 21, and if the unmanned conveying machine 20 is a forklift equipped with a rotation mechanism that rotates the fork 21, it is preferable that the conveying means information includes information related to the rotation angle of the fork 21. Also, for example, if the unmanned transport vehicle 20 is an unmanned transport vehicle (AGV) equipped with a liftable stage, it is preferable that the transport means information includes information on the lifting height of the stage, and if the unmanned transport vehicle is equipped with a robot arm, it is preferable that the transport means information includes information indicating the state of the robot arm. The state of the unmanned transport vehicle 20 represented by these pieces of information is required to transition from a state included in Xsub to a state also included in Xsub when the set of all states of the unmanned transport vehicle 20 (including continuous elements that cannot be handled in a Markov decision process) is X, and a set that is a subset of X and has a finite number of elements is Xsub, and to transition to another state after the transition (i.e., not to a dead end). In this way, by extensively including information related to the loading device in the transport means information, the phase space related to the state s in the Markov decision process can be made high-dimensional.

Reference Signs List 1: Incoming/Outgoing Management System 10: Incoming/Outgoing Management Device 11: Information Acquisition Unit 12: Memory Unit 13: Action Decision Unit 14: Loading/Unloading Command Unit 20: Automated Guided Vehicle 21: Fork 30: Rack 31: Shelf N: Baggage S: Warehouse T: External Terminals R1 to R3: Receiving Locations K1 to K60, S1 to S3: Storage Locations T1 to T3: Temporary Storage Locations F1 to F3: Shipping Location

Claims (6)

an information acquisition unit that acquires baggage information related to the baggage stored in the warehouse and transport means information related to a loading vehicle that is a transport means for transporting the baggage;
a behavior determination unit that determines a behavior of the cargo handling vehicle by performing reinforcement learning based on the baggage information and the transportation means information,
The information acquisition unit acquires, as the package information, first package information relating to packages that have arrived at the warehouse and second package information relating to packages that are scheduled to arrive at the warehouse;
The first package information includes information indicating a package location, information related to a scheduled shipping time when the package will be shipped, and information related to a shipping location when the package will be shipped,
The second package information includes information on a scheduled arrival time of the package and information on a place where the package will be delivered,
The transportation means information includes information related to a current location of the cargo handling vehicle,
The behavior decision unit determines the behavior of the loading/unloading vehicle that maximizes the reward when a package is transported to a shipping location of the package at the scheduled shipping time of the package, and when another package is present at the receiving location of the package at the scheduled arrival time of the package, the reward is reduced. an information acquisition unit that acquires baggage information related to the baggage stored in the warehouse and transport means information related to a loading vehicle that is a transport means for transporting the baggage;
a behavior determination unit that determines a behavior of the cargo handling vehicle by performing reinforcement learning based on the baggage information and the transportation means information,
The package information includes information indicating a package location, information related to a scheduled shipping time when the package will be shipped, and information related to a shipping location when the package will be shipped,
The transport means information includes information related to a current location of the cargo handling vehicle, information related to a direction of the cargo handling vehicle, and information related to a cargo handling lift height of the cargo handling vehicle,
The behavior decision unit decides the behavior of the loading and unloading vehicle that maximizes the reward, assuming that the reward is obtained when the cargo is transported to the shipping location of the cargo at the scheduled shipping time of the cargo.
A warehouse entry/exit management system characterized by the above . The incoming/outgoing management system according to claim 1 or 2, characterized in that the action decision unit decides on an action of the loading/unloading vehicle that maximizes the reward, assuming that the reward is reduced when the cargo is not transported to the shipping location of the cargo at the scheduled shipping time of the cargo. The receiving/shipping management system according to claim 1 , wherein the second package information further includes information on a scheduled shipping time when the package is to be shipped, and information on a shipping location where the package is to be shipped. The incoming/outgoing management system according to any one of claims 1 to 4, characterized in that the behavior decision unit determines, as the behavior of the loading vehicle, a storage location of the luggage in the warehouse, a transport timing of the luggage from the receiving location to the storage location, and a transport timing of the luggage from the storage location to the shipping location. a loading command unit that commands the loading vehicle to take the action determined by the action determination unit;
The receiving/shipping management system according to any one of claims 1 to 5, characterized in that the cargo handling vehicle is an unmanned guided vehicle that transports cargo based on instructions from the cargo handling command department.