CN112712251B - Ship intelligent scheduling method applied to barge management system - Google Patents

Ship intelligent scheduling method applied to barge management system Download PDF

Info

Publication number
CN112712251B
CN112712251B CN202011583873.1A CN202011583873A CN112712251B CN 112712251 B CN112712251 B CN 112712251B CN 202011583873 A CN202011583873 A CN 202011583873A CN 112712251 B CN112712251 B CN 112712251B
Authority
CN
China
Prior art keywords
ship
scheduling
barge
management system
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011583873.1A
Other languages
Chinese (zh)
Other versions
CN112712251A (en
Inventor
钟振洋
李小华
孙球喜
崔峰赫
郑东虹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuhai Port Information Technology Co ltd
Original Assignee
Zhuhai Port Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuhai Port Information Technology Co ltd filed Critical Zhuhai Port Information Technology Co ltd
Priority to CN202011583873.1A priority Critical patent/CN112712251B/en
Publication of CN112712251A publication Critical patent/CN112712251A/en
Application granted granted Critical
Publication of CN112712251B publication Critical patent/CN112712251B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/083—Shipping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
    • G06Q30/0635—Processing of requisition or of purchase orders

Abstract

The invention discloses a ship intelligent scheduling algorithm applied to a barge management system, which comprises the following operation steps: s1: and establishing a port channel network diagram based on a Neo4J database, storing wharf and channel information in a barge management system into a non-relational database Neo4J as nodes and directed edges in the directed graph, wherein the channel information needs to comprise directions, lengths, channel water depths, maximum ship heights and the like, and the information is stored as attributes of the directed edges and is used as an influencing factor of order matching in the subsequent scheduling, and expressing all geographic information in the barge management system in the form of directed graph node space. The intelligent ship scheduling algorithm applied to the barge management system integrates the route, the ship and the order, and obtains a relatively mature algorithm through continuous training of a model, so that the intelligent ship scheduling algorithm is used for scheduling decisions in actual production scenes, and scheduling efficiency and decision benefits are improved.

Description

Ship intelligent scheduling method applied to barge management system
Technical Field
The invention relates to the field of barge transportation, in particular to an intelligent ship scheduling algorithm applied to a barge management system.
Background
In the barge transportation field, the traditional barge transportation scheduling mode completely depends on the working experience of scheduling personnel, because the influence factors of ship scheduling are more, and meanwhile, the selectable schemes are quite many in each scheduling, so that the method has no standard how to judge the advantages and disadvantages of one scheduling result, mainly depends on personal experience, and needs to manually schedule the execution plan of each voyage of each ship.
Disclosure of Invention
The invention mainly aims to provide a ship intelligent scheduling algorithm applied to a barge management system, which can effectively solve the problems in the background technology.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
an intelligent scheduling algorithm for ships applied to a barge management system comprises the following operation steps:
s1: establishing a port channel network diagram based on a Neo4J database, storing wharf and channel information in a barge management system into a non-relational database Neo4J as nodes and directed edges in the directed graph, wherein the channel information needs to comprise directions, lengths, channel water depths, maximum ship heights and the like, and the information is stored as attributes of the directed edges and is used as an influencing factor of order matching in the subsequent scheduling, and expressing all geographic information in the barge management system in the form of directed graph node space to match ships, orders and routes;
s2: based on hierarchical reinforcement learning, an intelligent ship scheduling model is established, and a ship scheduling system is divided into two parts, and mainly comprises the following steps: the system comprises an offline learning module and an online matching module, wherein the offline learning module extracts information such as ship state, scheduling strategy and corresponding rewards from a system historical scheduling track, trains the information and updates parameters of a network, and simultaneously, a strategy value function V(s) obtained in the learning module is put into real-time storage, and the online matching module obtains the strategy value function V(s) from the real-time storage and calculates corresponding utility index rho corresponding to multiple orders of the ship ij Taking the index as the weight in the ship-order matching bipartite graph to carry out ship scheduling;
s3: the method is characterized by comprising the steps of defining a state, taking historical data of ship-order matching as a data source for extracting state characteristics, specifically taking factors affecting matching in the last period of time as the state of the current ship periphery when a batch of orders are transported, enabling s to represent the state, and enabling s to be s t ＝(l t ,μ t ,υ t ) Wherein the context feature vector v t Includes two kinds, dynamicCharacteristic v dt And a static characteristic v st ；
S4: after the reward function setting and determining the state space s and the action space a, the state transfer function t is also determined, and the reward function R in reinforcement learning is defined at the moment /r The rules of (2) are: when the allocation of one order is established, the reward value is matched with the single ship allocation rate of the ship for the order, otherwise, the reward value is 0, and the reward plastic method is utilized to enrich the expression of the reward function, and the reward of selecting the action a on the state s and transferring to the state s' is defined as:
R(s,o,s’)＝R 0 (s,o,s’)+Φ(s)
wherein R is 0 (s, o, s') is the originally defined reward function, Φ(s) is the potential function, and is a sub-objective in the vessel learning process, the potential function Φ(s) is defined as:
s5: the algorithm flow is based on the method for estimating the value function of reinforcement learning, a deep neural network is used for fitting, simulated scheduling is carried out, a simulation environment is initialized first, a ship-order matching method is continuously called before the task is finished to randomly execute scheduling, ship scheduling ideas in each scheduling are different, a reward value is returned after execution is finished to reflect the advantages and disadvantages of the current decision, and the reward value is continuously accumulated to reflect the advantages and disadvantages of all matching in the task execution process.
Preferably, the influencing factors in the step S3 include a dock location l where the ship is located t Original timestamp μ t Given context feature vector v t Etc.
Preferably, the dynamic characteristic v in the step S3 dt Real-time characteristics including supply and demand of given time-space points, and static characteristics v st The method comprises the steps of ship static state, current ship transportation state, information of transportation orders, cargo quantity and the like.
Preferably, the action space a in the step S4 includes a space where the ship can select a scheduling policy, that is, the order information that can be selected.
Compared with the prior art, the invention has the following beneficial effects:
in the invention, a ship scheduling model is established by using layered reinforcement learning, a route, a ship and an order are integrated, and a relatively mature algorithm is obtained by continuously training the model, so that the method is used for scheduling decisions in actual production scenes, and scheduling efficiency and decision benefit are improved.
Drawings
FIG. 1 is a flow chart of the deployment of the present invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention relates to a ship intelligent scheduling algorithm applied to a barge management system, which comprises the following operation steps:
s1: establishing a port channel network diagram based on a Neo4J database, storing wharf and channel information in a barge management system into a non-relational database Neo4J as nodes and directed edges in the directed graph, wherein the channel information needs to comprise directions, lengths, channel water depths, maximum ship heights and the like, and the information is stored as attributes of the directed edges and is used as an influencing factor of order matching in the subsequent scheduling, and expressing all geographic information in the barge management system in the form of directed graph node space to match ships, orders and routes;
s2: based on hierarchical reinforcement learning, an intelligent ship scheduling model is established, and a ship scheduling system is divided into two parts, and mainly comprises the following steps: the offline learning module extracts information such as ship state, scheduling strategy and corresponding rewards from a system historical scheduling track, trains and updates the informationThe parameters of the network, at the same time, the strategy value function V(s) obtained in the learning module is put into the real-time storage, and in the online matching module, the strategy value function V(s) is obtained from the real-time storage and the corresponding utility index rho corresponding to the ship and multiple orders is calculated ij Taking the index as the weight in the ship-order matching bipartite graph to carry out ship scheduling;
s3: the method comprises the steps of defining a state, taking historical data of ship-order matching as a data source for extracting state characteristics, specifically taking factors affecting matching in the last period of time as the state of the current ship periphery when a batch of orders are transported, wherein the influencing factors comprise the dock position l of the ship t Original timestamp μ t Given context feature vector v t Let s represent the state, let s be t ＝(l t ,μ t ,υ t ) Wherein the context feature vector v t Comprises two types, namely, dynamic characteristic v dt And a static characteristic v st Dynamic characteristic v dt Real-time characteristics including supply and demand of given time-space points, and static characteristics v st The method comprises the steps of static ship, current ship transportation state, information of transportation orders, cargo quantity and the like;
s4: after the state space s and the action space a are determined, the state transfer function t is also determined, the action space a comprises a space in which a ship can select a scheduling strategy, namely, selectable order information, and a reward function R in reinforcement learning is defined at the moment /r The rules of (2) are: when the allocation of one order is established, the reward value is matched with the single ship allocation rate of the ship for the order, otherwise, the reward value is 0, and the reward plastic method is utilized to enrich the expression of the reward function, and the reward of selecting the action a on the state s and transferring to the state s' is defined as:
R(s,o,s’)＝R 0 (s,o,s’)+Φ(s)
wherein R is 0 (s, o, s') is the originally defined reward function, Φ(s) is the potential function, and is a sub-objective in the vessel learning process, the potential function Φ(s) is defined as:
s5: the algorithm flow is based on the method for estimating the value function of reinforcement learning, a deep neural network is used for fitting, simulated scheduling is carried out, a simulation environment is initialized first, a ship-order matching method is continuously called before the task is finished to randomly execute scheduling, ship scheduling ideas in each scheduling are different, a reward value is returned after execution is finished to reflect the advantages and disadvantages of the current decision, and the reward value is continuously accumulated to reflect the advantages and disadvantages of all matching in the task execution process.
According to the invention, a ship scheduling model is established by using layered reinforcement learning, a route, a ship and an order are integrated, and a relatively mature algorithm is obtained by continuously training the model, so that the method is used for scheduling decisions in actual production scenes, and scheduling efficiency and decision benefit are improved.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (2)

1. An intelligent ship scheduling method applied to a barge management system is characterized in that: the method comprises the following operation steps:
s1: establishing a port channel network diagram based on a Neo4J database, storing wharf and channel information in a barge management system into a non-relational database Neo4J as nodes and directed edges in the directed graph, wherein the channel information needs to comprise directions, lengths, channel water depths and maximum ship heights, and the information is stored as attributes of the directed edges and is used as an influencing factor of order matching in the subsequent scheduling, and expressing all geographic information in the barge management system in the form of directed graph node space to match ships, orders and routes;
s2: establishing an intelligent ship scheduling model based on hierarchical reinforcement learning, and scheduling the shipThe system is divided into two parts, mainly comprising: the system comprises an offline learning module and an online matching module, wherein the offline learning module extracts ship states, scheduling strategies and corresponding rewarding information from a system historical scheduling track, trains the ship states and the scheduling strategies, updates parameters of a network, and simultaneously stores a strategy value function V(s) obtained in the learning module in real time, obtains the strategy value function V(s) from the real-time storage in the online matching module, and calculates corresponding utility indexes rho corresponding to multiple orders of the ship ij Taking the index as the weight in the ship-order matching bipartite graph to carry out ship scheduling;
s3: the method is characterized by comprising the steps of defining a state, taking historical data of ship-order matching as a data source for extracting state characteristics, specifically taking factors affecting matching in the last period of time as the state of the current ship periphery when a batch of orders are transported, enabling s to represent the state, and enabling s to be s t ＝(l t ,μ t ,υ t ) Wherein the context feature vector v t Comprises two types, namely, dynamic characteristic v dt And a static characteristic v st The influencing factors include the dock location l at which the ship is located t Original timestamp μ t Given context feature vector v t Dynamic characteristic v dt Including real-time characteristics of supply and demand for a given spatiotemporal point, static characteristics v st The method comprises the steps of static ship, current ship transportation state, information of transportation orders and cargo quantity;
s4: after the reward function setting and determining the state space s and the action space a, the state transfer function t is also determined, and the reward function R in reinforcement learning is defined at the moment /r The rules of (2) are: when the allocation of one order is established, the rewarding value is matched with the single ship allocation rate of the ship for the order, otherwise, the rewarding value is 0, the rewarding plastic method is utilized to enrich the expression of the rewarding function, and the action space a is selected on the state space s and transferred to the state space s ’ "rewards are defined as:
R(s,o,s ’ )＝R 0 (s,o,s ’ )+Φ(s)
wherein R is 0 (s,o,s ’ ) Is thatThe originally defined reward function, Φ(s), is a potential function, and as a sub-target in the ship learning process, the potential function Φ(s) is defined as:
s5: the algorithm flow is based on the method for estimating the value function of reinforcement learning, a deep neural network is used for fitting, simulated scheduling is carried out, a simulation environment is initialized first, a ship-order matching method is continuously called before the task is finished to randomly execute scheduling, ship scheduling ideas in each scheduling are different, a reward value is returned after execution is finished to reflect the advantages and disadvantages of the current decision, and the reward value is continuously accumulated to reflect the advantages and disadvantages of all matching in the task execution process.
2. The intelligent scheduling method for ships in a barge management system according to claim 1, wherein: the action space a in the step S4 includes a space where the ship can select a scheduling policy, i.e., selectable order information.
CN202011583873.1A 2020-12-28 2020-12-28 Ship intelligent scheduling method applied to barge management system Active CN112712251B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011583873.1A CN112712251B (en) 2020-12-28 2020-12-28 Ship intelligent scheduling method applied to barge management system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011583873.1A CN112712251B (en) 2020-12-28 2020-12-28 Ship intelligent scheduling method applied to barge management system

Publications (2)

Publication Number Publication Date
CN112712251A CN112712251A (en) 2021-04-27
CN112712251B true CN112712251B (en) 2023-09-12

Family

ID=75545933

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011583873.1A Active CN112712251B (en) 2020-12-28 2020-12-28 Ship intelligent scheduling method applied to barge management system

Country Status (1)

Country Link
CN (1) CN112712251B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109375235A (en) * 2018-11-30 2019-02-22 武汉理工大学 Inland navigation craft freeboard detection method based on deeply neural network
CN111033535A (en) * 2018-06-05 2020-04-17 北京嘀嘀无限科技发展有限公司 System and method for bus order scheduling

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523151A (en) * 2018-11-09 2019-03-26 上海港引航站 A kind of yard craft pilotage intelligent dispatching method based on data-driven
CN111845741B (en) * 2020-06-28 2021-08-03 江苏大学 Automatic driving decision control method and system based on hierarchical reinforcement learning
CN111694365B (en) * 2020-07-01 2021-04-20 武汉理工大学 Unmanned ship formation path tracking method based on deep reinforcement learning
US20200356834A1 (en) * 2020-07-27 2020-11-12 Zhu ZHOU Hierarchical reinforcement learning algorithm for nfv server power management

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111033535A (en) * 2018-06-05 2020-04-17 北京嘀嘀无限科技发展有限公司 System and method for bus order scheduling
CN109375235A (en) * 2018-11-30 2019-02-22 武汉理工大学 Inland navigation craft freeboard detection method based on deeply neural network

Also Published As

Publication number Publication date
CN112712251A (en) 2021-04-27

Similar Documents

Publication Publication Date Title
Chen et al. Forecasting container throughputs at ports using genetic programming
Cohon et al. Multiobjective screening models and water resource investment
CN110222938B (en) Short-term peak-load regulation scheduling collaborative optimization method and system for cascade hydropower station group
CN104217258B (en) A kind of electric load sigma-t Forecasting Methodology
CN110837915B (en) Low-voltage load point prediction and probability prediction method for power system based on hybrid integrated deep learning
CN112836974B (en) Dynamic scheduling method for multiple field bridges between boxes based on DQN and MCTS
CN112488452B (en) Energy system management multi-time scale optimal decision method based on deep reinforcement learning
CN104866924A (en) Active power distribution network planning and operation combined optimization method
CN106096771A (en) The parameter selection method of support vector machine and system and load forecasting method and system
CN104077634A (en) Active-reactive type dynamic project scheduling method based on multi-objective optimization
CN109214565A (en) A kind of subregion system loading prediction technique suitable for the scheduling of bulk power grid subregion
CN115334106A (en) Microgrid transaction consensus method and system based on Q method and power grid detection and evaluation
CN107330538B (en) Method for compiling reservoir adaptive scheduling rules under climate change condition
Hakimi-Asiabar et al. Multi-objective genetic local search algorithm using Kohonen’s neural map
CN109325610B (en) Piecewise linear scheduling method for multi-water-source cascade pump station water transfer system
CN112712251B (en) Ship intelligent scheduling method applied to barge management system
CN107871157B (en) Data prediction method, system and related device based on BP and PSO
CN110852808A (en) Asynchronous adaptive value evaluation method of electronic product based on deep neural network
CN106228235A (en) A kind of land utilization space Optimal Configuration Method taking general layout and PROCESS COUPLING effect into account
CN117117878A (en) Power grid demand side response potential evaluation and load regulation method based on artificial neural network and multi-agent reinforcement learning
Haddad et al. Dynamic penalty function as a strategy in solving water resources combinatorial optimization problems with honey-bee mating optimization (HBMO) algorithm
CN115965145A (en) Intelligent parking indicating method based on business rule engine
Dong et al. QCs scheduling scheme of genetic algorithm (GA) and improved firefly algorithm (FA)
Notsu et al. Simple reinforcement learning for small-memory agent
CN114861995A (en) Demand prediction method and device for power supplies and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant