KR20230067113A - A method for calculating the estimated time of arrival of a ship and an device for the same - Google Patents

Abstract

The present invention relates to a method for calculating an estimated arrival time of a ship and a calculation device thereof, and more specifically to a method for reinforcing learning of past ship routes and using a result obtained as a result of learning in this way to predict the navigation status of ships currently in operation, ultimately calculating the final estimated arrival time of the ship more accurately, and a calculation device thereof.

Description

A method for calculating the estimated time of arrival of a ship and an arithmetic device therefor

The present invention relates to a method and an arithmetic device for calculating an estimated time of arrival of a ship, and more specifically, to reinforce and learn the course of a ship in the past, and use the result obtained as a result of this learning to predict the navigation situation of a ship currently in operation. Ultimately, it relates to a method and an arithmetic device that more accurately calculates the final estimated time of arrival of a ship.

Estimated time of arrival (ETA) in the maritime transportation industry is generally obtained by simply calculating when a ship will arrive at its destination when moving at an average speed, using only the relational expression between distance and speed. However, recently, with the development of technology, attempts have been made to reflect the influence of various variables in calculating the estimated time of arrival, and such efforts are expected to continue in the future.

However, it is still a very difficult task to obtain an accurate estimated time of arrival, as the route and speed that the actual vessel will operate are inevitably affected by countless variables such as weather, season, wind, and port congestion.

The present invention is intended to propose a method and system that enables immediate route calculation and estimated arrival time calculation by reflecting in real time the effects that a ship may receive when moving to a destination along a route, such as past ship route learning It relates to a method and system for accurately predicting the arrival time of a ship compared to the prior art by learning data using a reinforcement learning method and correcting the estimated time of arrival by referring to the learned result.

Republic of Korea Patent Publication No. 10-2021-0066108 A (2021.06.07. Publication)

An object of the present invention is to learn past ship operation information through a reinforcement learning algorithm and to store the learned result values in a database so that they can be used for calculating the real-time estimated arrival time of a ship in operation.

Through this, it is possible to calculate the ETA more accurately compared to the conventional calculation of the ETA by simply quantitatively reflecting the surrounding environment information, and ultimately raise the accuracy to 80% or more, thereby reducing the ETA error. The purpose is to reduce unnecessary costs and waste of manpower.

In addition, an object of the present invention is to significantly reduce resources consumed during maritime transportation by not only accurately predicting the expected arrival time but also selecting the most efficient route from the departure point to the destination to the current situation.

On the other hand, the technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

The present invention is intended to solve the above problems, and a method for calculating an estimated time of arrival of a ship according to the present invention includes the steps of collecting location information about a ship at predetermined intervals; generating a first route from a starting point to a destination point by referring to at least one dynamic environment variable; generating a second route by at least partially modifying the first route by referring to a ship operation database; determining the second route as a final route when the generated second route satisfies a predetermined condition; and calculating an estimated time of arrival of the vessel based on the final route.

In addition, in the method for calculating the estimated time of arrival of the ship, the dynamic environment variables include meteorological variables related to weather information, wave height variables related to wave heights, oil price variables related to international oil prices, accident variables related to marine accidents, or traffic volume related to marine traffic. At least one of the variables may be included.

In addition, the method for calculating the estimated time of arrival of the ship further includes generating at least one basic route from the starting point to the ending point before generating the first route, wherein the first route comprises: It may be characterized in that it is created by applying at least one dynamic environment variable to a basic path.

In addition, in the method for calculating the expected arrival time of the ship, the ship operation database may be a database in which result values obtained by reinforcement learning of past ship operation information are accumulated and stored.

At this time, the result values accumulated and stored in the ship operation database are expected when the ship moves from the s-th node to the s+1-th node (s is a natural number of 1 or more) in an arbitrary path composed of a plurality of nodes. It may be characterized in that they are result values calculated by iterative operation of a function aimed at maximizing compensation.

Also at this time, the function

로 정의되되, s는 노드상태, a는 액션, γ는 할인율, Q(s,a)는 특정 노드상태 s에서 액션 a를 취할 경우 받을 보상의 기대값을 산출하기 위한 행동가치함수, a’은 다음 노드상태(s’)에서 행동가치함수의 값을 최대로 만드는 액션인 것을 특징으로 할 수 있다.

, where s is the node state, a is the action, γ is the discount rate, Q(s,a) is the action value function for calculating the expected value of the reward received when action a is taken in the specific node state s, and a' is It may be characterized as an action that maximizes the value of the action value function in the next node state (s').

Also, at this time, the action value function is

로 정의되되, s는 노드상태, a는 액션, E는 기대값, R은 보상, γ는 할인율, t는 시간이고, Q_π(s,a)는 특정 상태 s에서 액션 a를 취할 경우 받을 보상의 기대값을 산출하기 위한 함수인 것을 특징으로 할 수 있다.

, where s is the node state, a is the action, E is the expected value, R is the reward, γ is the discount rate, t is the time, and Q _π (s,a) is the reward received when action a is taken in a specific state s It may be characterized in that it is a function for calculating the expected value of .

In addition, in the method for calculating the estimated time of arrival of the ship, the step of generating the second route may be performed by an artificial intelligence algorithm, and the artificial intelligence algorithm may be reinforced and learned based on a ship operation database.

In addition, in the method for calculating the estimated time of arrival of the ship, the generating of the second route is characterized in that a plurality of second routes are generated according to which one of the plurality of input variables is weighted. In this case, the final route may be characterized in that the second route having the lowest cost required for navigation among the plurality of second routes is determined.

On the other hand, according to another embodiment of the present invention, a method for reinforcement learning of an algorithm for generating a ship's path includes loading past ship operation information; and repeatedly executing a function for calculating an expected value of compensation according to a navigation result from a start point to a destination point of a ship for a plurality of virtual routes in the past ship operation information.

In the reinforcement learning method for the algorithm for generating the path of the ship, the path from the starting point to the destination point includes a plurality of nodes, and the step of repeatedly executing the function is that the ship moves from the s-th node to the s+1-th node. It may be characterized in that it is a step of repeatedly executing a function aimed at maximizing an expected reward when moving to a node (s is a natural number of 1 or more).

In addition, the reinforcement learning method of the algorithm for generating the ship's path repeatedly executes the function for the plurality of virtual paths, but in each iterative execution step, different variables are applied to maximize the expected value of the reward It can be characterized by doing.

The reinforcement learning method for the algorithm for generating the ship's path includes the steps of selecting virtual paths whose expected value of compensation calculated as a result of repeatedly executing a function for the plurality of virtual paths is equal to or greater than a reference value and storing them in a database. can include more.

On the other hand, the calculation device for calculating the expected time of arrival of the ship according to another embodiment of the present invention includes an information collection unit for collecting location information about the ship or information related to dynamic environment variables; a calculation unit for calculating a path from a starting point to a destination of the ship by referring to at least one dynamic environment variable or a ship operation database; a determination unit which determines one of the plurality of paths calculated by the calculation unit as a final path; a dynamic environment database in which information related to dynamic environment variables is stored; and a control unit controlling the information collection unit, the calculation unit, the determination unit, and the dynamic environment database.

In addition, the calculation device may further include a ship operation database for storing past ship operation information.

In addition, in the calculation device, the calculation unit generates a first route from the start point to the destination point of the ship by referring to at least one or more dynamic environment variables, and at least partially corrects the first route by referring to the ship operation database. It may be characterized by generating one second path.

In addition, in the calculation device, the ship operation database accumulates and stores result values obtained by reinforcement learning of past ship operation information, and the result values are determined by the ship in an arbitrary path composed of a plurality of nodes. It may be characterized in that they are result values calculated by iterative operation of a function aimed at maximizing an expected reward when moving from the s node to the s+1th node (s is a natural number of 1 or greater).

In addition, in the arithmetic device, the function is

로 정의되되, s는 노드상태, a는 액션, γ는 할인율, Q(s,a)는 특정 노드상태 s에서 액션 a를 취할 경우 받을 보상의 기대값을 산출하기 위한 행동가치함수, a’은 다음 노드상태(s’)에서 행동가치함수의 값을 최대로 만드는 액션인 것을 특징으로 할 수 있다.

, where s is the node state, a is the action, γ is the discount rate, Q(s,a) is the action value function for calculating the expected value of the reward to be received when action a is taken in the specific node state s, and a' is It may be characterized as an action that maximizes the value of the action value function in the next node state (s').

Also, at this time, the action value function is

로 정의되되, s는 노드상태, a는 액션, E는 기대값, R은 보상, γ는 할인율, t는 시간이고, Q_π(s,a)는 특정 상태 s에서 액션 a를 취할 경우 받을 보상의 기대값을 산출하기 위한 함수인 것을 특징으로 할 수 있다.

, where s is the node state, a is the action, E is the expected value, R is the reward, γ is the discount rate, t is the time, and Q _π (s,a) is the reward received when action a is taken in a specific state s It may be characterized in that it is a function for calculating the expected value of .

According to the present invention, there is an effect of calculating the estimated time of arrival of the ship more accurately.

In addition, since the expected arrival time of the ship can be accurately predicted, logistics operations performed at the port can be performed on time, and thus, unnecessary costs in the entire logistics process can be minimized.

On the other hand, the effects of the present invention are not limited to those mentioned above, and other technical effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a simplified diagram for easy understanding of results obtained by using a method for calculating an estimated time of arrival of a ship according to the present invention.
2 shows a method for calculating the estimated time of arrival of a ship according to the present invention in sequence.
FIG. 3 illustrates an example of a route map referred to in the step of generating a second route among the steps of the calculation method.
4 is a conceptual diagram for understanding the principle of the reinforcement learning algorithm to be utilized in the present invention.
5 illustrates an ETA calculation method according to another embodiment of the present invention.
6 shows detailed configurations of an arithmetic device according to the present invention.

Objects and technical configurations of the present invention and details of the operational effects thereof will be more clearly understood by the following detailed description based on the accompanying drawings in the specification of the present invention. An embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

The embodiments disclosed herein should not be construed or used as limiting the scope of the present invention. It goes without saying that the description, including the embodiments herein, has a variety of applications for those skilled in the art. Therefore, any embodiments described in the detailed description of the present invention are illustrative for better explaining the present invention, and the scope of the present invention is not intended to be limited to the examples.

The functional blocks shown in the drawings and described below are only examples of possible implementations. Other functional blocks may be used in other implementations without departing from the spirit and scope of the detailed description. Also, while one or more functional blocks of the present invention are represented as separate blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

In addition, the expression that certain components are included simply refers to the existence of the corresponding components as an expression of “open type”, and should not be understood as excluding additional components.

Furthermore, it should be understood that when a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. do.

1 is an example of a situation that a user may experience when using the method for calculating the estimated time of arrival of a ship according to the present invention. it is easily illustrated.

As mentioned above, the ultimate purpose of the present invention is to more accurately predict the expected arrival time of a ship, which is a means of maritime transportation, and among the detailed steps of this method, a step of referring to data learned by reinforcement learning is included.

Figure 1 shows routes that can be expected when a specific ship A transports cargo from a departure point (S) to a destination (D), and the expected arrival time is calculated and described for each route. For reference, the subject that calculates the ship's route and estimated time of arrival will be an arithmetic device to be described later, and the arithmetic device may include both a central processing unit and any device equipped with a memory. First of all, the route ① shows the basic route, and shows the route from the departure point to the destination along the basic route that the ship can navigate without considering other variables. In addition, the expected arrival time corresponding to route ① is also calculated by referring to the average speed of the ship, the total length of the route, and, if necessary, the anchoring time at the stopping point. On the other hand, route ② is created by reflecting several variables by the computing device, and the types of variables reflected at this time may include weather, wave height, oil price, marine accidents, or marine traffic. In other words, the path ② reflects the above external environmental factors, and it can be said that the accuracy is higher than the previous path ①. Finally, route ③ is modified by referring to the database built by prior reinforcement learning in route ②, and it can be said to be the most important route generation step among the methods for calculating the expected arrival time of the ship according to the present invention. The route ③ was created through a correction process to reduce the error of route ② in the state where a plurality of past ship operation information was learned by an artificial intelligence algorithm, especially an artificial intelligence algorithm that adopts a reinforcement learning method. ③ has higher accuracy than path ②. For reference, in FIG. 1, the estimated time of arrival of the ship corresponding to route ③ is 20:00 on October 31, 2021, and the estimated time of arrival corresponding to route ② is 9:00 on November 1, 2021, and the basic route route ① The corresponding expected arrival time is listed as 15:00, and from this, it can be confirmed that the route ③ that reflects the result using the artificial intelligence algorithm is the route that can complete the maritime transportation in the fastest time with the highest accuracy. In other words, when the method for calculating the estimated time of arrival of a ship according to the present invention is used, as a result, it is possible to accurately predict the expected time of arrival of the ship and to derive the shortest maritime travel time of the ship, so that various fields of the maritime transportation business can be obtained. A positive effect is expected from For reference, routes ①, ②, and ③ shown in FIG. 1 will be named “basic route”, “first route”, and “second route” in the detailed description, respectively, unless otherwise specified. It is understood that the is used as it is.

2 shows a method for calculating the estimated time of arrival of a ship according to the present invention in sequence. As mentioned above, this calculation method can be performed by an arithmetic device to be described later. In the following, each step will be examined in more detail.

Referring to FIG. 2 , the method for calculating the estimated time of arrival of the ship may first start with the step of collecting location information about the ship at predetermined intervals (S10). Ship location information can be collected through various sources. For example, ship location information can be obtained through network access to a database operator that provides international ship location tracking services such as Marine Traffic or Vessel Finder. On the other hand, the predetermined period (period) can be determined according to the setting of the user (or the manager who manages the computing device), the user can set to collect the location information of the currently operating ship every 6 hours, for example. In addition, the location information of the ship collected in this step will preferably be referred to as GPS information, but is not necessarily limited thereto.

After the step of collecting the location information of the ship, the calculation device may generate at least one basic route from the starting point to the destination point (S20). The default path, simply put, refers to a path in an initialized state, and may mean a path that will be subject to operation or modification by an algorithm in the future. The basic route may be pre-determined for each start-and-end point pair in which a start point and an end point are determined. For example, a default route such as route ① in FIG. 1 is pre-determined between Busan (Korea) and Vancouver (Canada). There may be. Alternatively, if it is not determined in advance, one of the routes determined from the starting point to the destination point may be arbitrarily determined and defined as a basic route. For reference, the above embodiments are only one example of generating a basic route in step S20, but are not necessarily limited thereto, and it is sufficient if the basic route necessary for starting the method for calculating the expected arrival time of a ship according to the present invention can be defined. will be.

After the basic route is created, a first route from the starting point to the destination point may be created with reference to at least one or more dynamic environment variables (S30). Dynamic environmental variables refer to factors that can affect the operation of a ship at sea, and the dynamic environmental variables preferably represent a specific value or specific range in numbers, or characters may be represented by In addition, various types of dynamic environment variables may exist, but weather variables related to weather information, wave height variables related to wave heights, oil price variables related to international oil prices, accident variables related to marine accidents, or traffic volume variables related to marine traffic can be included by default. The above dynamic environmental variables can also be collected through an online network regardless of the type of computing device. It is possible to access and collect information, and the information collected in this way can be built into a database in the computing device itself again by the computing device or in a cloud storage space accessible by the computing device. On the other hand, in the first route considering dynamic environmental variables, for example, in relation to weather variables, some of the routes are modified so as not to directly pass through areas where heavy rain is expected, or in relation to oil price variables, some of the routes are basic routes. It can be understood to mean a route created through a process such as being modified to be shorter than , or partially lengthening the route so that port access is possible when the traffic volume around the stopover is below a certain level in relation to the traffic volume variable. Preferably, the first path generation process is implemented such that a relational expression for applying each dynamic environment variable exists, and a process in which each formula is reflected on the basic path is performed by algebraic or geometrical operation. can

Meanwhile, after the first route is generated, a second route obtained by at least partially modifying the first route may be generated by referring to the ship operation database (S40). That is, this step is a step of creating a second route by modifying the first route by referring to a separate dedicated database, more precisely, a ship operation database, even though there is already a first route with dynamic environment variables reflected, The step of generating the second path may be regarded as a major step of the calculation method according to the present invention.

Even in the first route with the dynamic environment variables reflected, the probability that the actual ship will navigate along the predicted first route is not high, and of course, the estimated arrival time of the ship calculated based on the first route is also less accurate. Therefore, in the present invention, the calculated first route is modified once more, but instead, the ship operation record is modified by referring to the ship operation database in which the result values learned by reinforcement learning are stored, so that the expected ship operation path and It was intended to predict the expected arrival time more accurately. That is, the second path may be a path in which result values learned by reinforcement learning are reflected.

At this time, the result value stored in the ship operation database is used to reinforce and learn the navigation information of past ships (or ships), but in an arbitrary navigation route consisting of a plurality of nodes, the ship is the first node at the s node. It may be a result value calculated by iterative operation of a function aimed at maximizing a reward that can be obtained when moving to node s+1 (where s is a natural number greater than or equal to 1). In this case, compensation can also be defined as a benefit that the user will obtain according to the operation of the ship. For example, the fact that the ship has safely arrived at the destination, the user's economic benefits obtained as the ship operates, etc. can be an example of Although these compensations are verbally explained to help the understanding of the invention, it will be understood that the compensations must be digitized in order to perform calculations according to actual algorithms.

On the other hand, the aforementioned function, that is, a function aimed at maximizing a reward that can be obtained when a ship at a certain node proceeds to the next node can be expressed by the following equation.

In the above equation, s is the node state, a is the action, γ is the discount rate, Q(s,a) is the action value function for calculating the expected value of the reward to be received when action a is taken in the specific node state s, and a' is the following It may mean an action that maximizes the value of the action value function in the node state (s'). In other words, S is the state where the ship is currently placed, and a may mean the movement of the ship.

In addition, the action value function included in Equation 1 above may be defined as follows.

Similarly, in the above equation, s is the node state, a is the action, E is the expected value, R is the reward, γ is the discount rate, t is the time, and Q _π (s,a) is the value received when action a is taken in a specific state s. It is a function that calculates the expected value of the reward.

3 is a diagram for easily understanding the algorithm defined by Equation 1 above. Assuming that a specific ship navigates along a route from a first node to a fifth node, motions that can be selected at each node, And the reward that can be obtained when the movement is selected is arbitrarily described.

First, when the ship is placed on the 4th node or the 3rd node, there is no choice about the action that the ship can take, that is, the movement, so the reward is always the same regardless of the state and action after the 3rd node. will be computed as

On the other hand, when the ship is in the 2nd node state, the ship can pass through the 2-1 node to reach the 3rd node, and pass through the 2-2 node to reach the 3rd node. You can choose one of the actions or actions that can directly reach the third node without going through a separate node. In each case, the rewards will be (0.5+0.1), (0.2+0.1), (0.7). At this time, since the selection to maximize the reward should be an action given a 0.7 reward, the ship movement directly proceeding to the third node will be selected in the second node state.

On the other hand, even when placed in the first node state, the ship can select one of the actions that can reach the second node through the 1-1 node and the action that can directly reach the second node. In the first node state, an action (reward 0.6) to proceed to the second node will be selected as soon as a larger reward is expected. At this time, when the ship is already in the second node state, it is confirmed that the maximum reward of 0.7 is expected, and the maximum reward that can be obtained by action (movement) in the first node state is 0.6. The reward expected to be obtained for reaching the third node from V would be 1.3.

Meanwhile, in this way, the operation according to the function of Equation 1 may be repeated, and the resultant value may be stored in the aforementioned ship operation database. At this time, the form of the stored result value may be a combination of text and numbers, but may have the form of a route map as shown in FIG. 4 in some cases.

4 shows an exemplary route map showing a route from the starting point S to the ending point D. Such route maps may be cumulatively stored in the ship operation database, and the second route In the step of generating, any one of the route maps stored in this way may be selected for reference and used to generate the actual second route. Each route map may have different conditions, such as the node state where the ship is located, the action (movement) that the ship can take to proceed to the next node state, and the compensation according to the action. Creating a second route In , a route map in a state similar to the state in which the ship is currently placed (including a state in which dynamic environment variables are considered) can be searched and the searched route map can be referred to.

For reference, in step S40 described above, when generating the second route, it has been explained that the second route can be generated by referring to the ship operation database in which the results of reinforcement learning are accumulated and stored. The algorithm itself may be implemented as an artificial intelligence algorithm. That is, in step S40, the second path is generated by an artificial intelligence algorithm, and at this time, the artificial intelligence algorithm may be in a state in which it has been learned by reinforcement learning of past ship operation information, and through this, directly the current state of the ship, A second route, which is a result value, can be calculated by taking an action that the current ship can take, an expected reward, and the like as inputs.

On the other hand, the operation in the process of constructing the ship operation database or the reinforcement learning algorithm in the process of directly generating the second route may be repeated multiple times, and each time it is repeated, the type of compensation is changed so that the user wants It is also possible to generate a path with a variable weighted state.

Meanwhile, after the second route is generated, the step of determining the final route (S50) and the step of calculating the estimated time of arrival (S60) may be executed. The final route may be determined according to whether the second route generated by the computing device satisfies a condition set by the user, and if the generated second route satisfies the condition, the corresponding second route becomes the final route, Otherwise, another route may be determined as the final route by returning to the step of generating the first route or the step of generating the second route again. Meanwhile, the estimated time of arrival will be calculated according to the final route after the final route is determined.

For reference, the processes listed in FIG. 2 may be repeatedly executed according to a user-determined period. For example, a step of considering a collection point as one node according to a cycle of collecting ship location information and generating a basic route, a first route, or a second route according to the state of the node may be performed.

In addition, for reference, the function used to calculate the result value, that is, the algorithm, can be learned through the iterative process of the function described above, which can be said to be the same for the algorithm that directly generates the second path. will be. Since the method of learning the algorithm is substantially the same as the process of calculating the result values in the ship operation database, a detailed description will be omitted.

Referring to FIG. 2 above, an embodiment of a method for calculating an estimated time of arrival of a ship according to the present invention has been looked at.

5 shows another embodiment of a method for calculating the estimated time of arrival of a ship according to the present invention. Step S200 of generating a default path in the embodiment of FIG. 5 corresponds to step S20 of FIG. 2, and thus a detailed description thereof will be omitted.

On the other hand, after step S200, a first route with reference to dynamic environment variables may be created on the computing device (S300), there may be a movement condition input with weights reflected (S400), and cost calculation for each node of each route (S450). ) can be made. For example, there may be a movement condition input in which weights are reflected, such as time or oil value, and the computing device may calculate the cost of each movement route for each node according to the weighted movement condition. Meanwhile, in this step, as mentioned above in the description of FIG. 2 , result values stored in the ship operation database may be referred to. Among them, the step of inputting the movement conditions with weights reflected (S400) and the step of calculating the cost for each node of each route (S450) is an exemplary detailed implementation of the step of generating the second route (S40) with reference to the ship operation database described in FIG. 2. can be understood as an example. In addition, in step S400 of FIG. 5, the interface may be implemented so that the user can arbitrarily input a specific movement condition, that is, the calculator receives input from the user, but more preferably automatically by an algorithm. Repetitive calculations can be performed by repeatedly inputting movement conditions in which specific weights are reflected. That is, the simulation by the algorithm can be made repeatedly and automatically.

After step S450, it is determined whether the moving route satisfies the minimum cost condition (S500), and according to the determination result, a final route is generated (S600), or another route is generated by returning to step S300 or step S400 again. can The step of determining whether the movement path satisfies the minimum cost condition and generating the final path accordingly can be understood as a detailed embodiment of step S50 in FIG. 2 . In this case, the term cost can be understood as including all costs required when the ship is operated along the corresponding movement route in a specific movement route calculated according to the execution of the algorithm, which includes, for example, ship operation and / Alternatively, various types of costs may be included, such as the cost of using manpower required for navigation, the cost of oil, the cost of docking in a port, or the cost of maintaining a ship. In addition, the step of determining whether a certain movement path satisfies the minimum cost condition in step S500 is, for example, a process of determining whether the cost is the lowest compared to other previously calculated movement routes, or a minimum cost arbitrarily set by the user. It can be implemented in various processes, such as a process of determining whether the cost of the travel path calculated in contrast has a larger value. In addition, the step S500 is not necessarily implemented only as a process of determining on the basis of cost, and in generating the final route, the weighted items determined by the user or other items necessary for considering the final route can be the criteria. to understand the point.

6 shows an arithmetic device 200 according to the present invention, and the arithmetic device 200, which is the subject of execution of the method for calculating the estimated time of arrival of a ship according to the present invention, is largely divided into an information collection unit 201 and a calculation unit 202. , a decision unit 203, and a control unit 206, and may further include a dynamic environment database 204 and a ship operation database 205.

The information collection unit 201 is a component that collects and stores information necessary for calculation according to the present invention, and is a component that collects information by accessing a subject that provides location information about a ship or information related to dynamic environment variables. In addition, the collected information is stored in the database so that it can be used in the future calculation process.

The calculation unit 202 is a component that calculates a route from a starting point to a destination point by referring to at least one dynamic environment variable and/or a ship operation database. The calculation unit 202 calculates a basic route, a first route, or A second route can be created. Since the process of generating each path has been described above, a detailed description thereof will be omitted.

The determination unit 203 is a component that determines one of the plurality of paths calculated by the calculation unit 202 as the final path, and in this determination, a reference value or the like previously designated by the user may exist.

The dynamic environment database 204, as the term implies, is a configuration in which the information collected by the information collection unit 201 is stored, and the ship operation database 205 is the result values followed by the reinforcement learning function, preferably a route map ( map) is a configuration that can be stored, and at this time, the function can be defined by Equations 1 and 2 described above.

Finally, the arithmetic device 200 may include a control unit 206, which may actually correspond to a central processing unit. The central processing unit may also be called a controller, a microcontroller, a microprocessor, a microcomputer, and the like. In addition, the central processing unit may be implemented by hardware, firmware, software, or a combination thereof. When implemented using hardware, an ASIC (application specific integrated circuit) or DSP (digital signal processor) , DSPD (digital signal processing device), PLD (programmable logic device), FPGA (field programmable gate array), etc., when implemented using firmware or software, modules, procedures, or functions that perform the above functions or operations Firmware or software may be configured to include. In addition, the service server 200 may also include a memory as a matter of course. The memory includes read only memory (ROM), random access memory (RAM), erasable programmable read only memory (EPROM), and electrically erasable programmable read (EEPROM) memory. -Only Memory), flash memory, SRAM (Static RAM), HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

Above, the ship estimated time of arrival calculation method and calculation device according to the present invention have been looked at. On the other hand, the present invention is not limited to the specific embodiments and application examples described above, and various modifications may be carried out by those skilled in the art to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, these modifications should not be understood separately from the technical spirit or perspective of the present invention.

200 arithmetic unit
201 Information collection department
202 calculation unit
203 Determination Department
204 Dynamic environment database
205 Ship Operation Database
206 Control

Claims (20)

In the method of calculating the estimated time of arrival of the ship,
Collecting location information about the vessel at a predetermined period;
generating a first route from a starting point to a destination point by referring to at least one dynamic environment variable;
generating a second route by at least partially modifying the first route by referring to a ship operation database;
determining the second route as a final route when the generated second route satisfies a predetermined condition; and
Calculating an estimated ship arrival time based on the final route;
including,
How to calculate the ship's expected arrival time.
According to claim 1,
The dynamic environment variable,
Characterized in that it includes at least one of weather variables related to weather information, wave height variables related to wave heights, oil price variables related to international oil prices, accident variables related to marine accidents, or traffic volume variables related to marine traffic,
Estimated arrival time calculation method.
According to claim 1,
Before the step of generating the first path,
Generating at least one basic path from the starting point to the destination point; further comprising,
Characterized in that the first path is generated by applying at least one dynamic environment variable to the basic path.
Estimated arrival time calculation method.
According to claim 1,
The ship operation database,
Characterized in that the database in which the results of reinforcement learning of past ship operation information are accumulated and stored,
Estimated arrival time calculation method.
According to claim 4,
The result values accumulated and stored in the ship operation database are,
In an arbitrary path composed of a plurality of nodes, when a ship moves from the s-th node to the s+1-th node (s is a natural number greater than or equal to 1), by iterative operation of a function aimed at maximizing the expected reward Characterized in that the calculated result values are
Estimated arrival time calculation method.
According to claim 5,
The function

defined as,
s is the node state, a is the action, γ is the discount rate, Q(s,a) is the action value function for calculating the expected value of the reward to be received when action a is taken in a specific node state s, and a' is the next node state ( Characterized in that it is an action that maximizes the value of the action value function in s'),
How to calculate the ship's expected arrival time.
According to claim 6,
The action value function is

defined as,
s is node state, a is action, E is expected value, R is reward, γ is discount rate, t is time,
Characterized in that Q _π (s, a) is a function for calculating the expected value of the reward to be received when action a is taken in a specific state s,
Estimated arrival time calculation method.
According to claim 1,
The step of generating the second path,
It is executed by an artificial intelligence algorithm, characterized in that the artificial intelligence algorithm is reinforced based on a ship operation database.
Estimated arrival time calculation method.
According to claim 1,
The step of generating the second path,
Characterized in that a plurality of second paths are generated according to which one of the plurality of input variables is weighted,
Estimated arrival time calculation method.
According to claim 9,
Characterized in that the final route is determined as a second route with the lowest cost required for navigation among the plurality of second routes,
Estimated arrival time calculation method.
In the reinforcement learning method for an algorithm for generating a ship's path,
loading past ship operation information; and
In the past ship operation information, repeatedly executing a function for calculating an expected value of compensation according to a navigation result from a starting point to a destination point of a ship for a plurality of virtual routes;
including,
A method for reinforcement learning of an algorithm for generating a ship's path.
According to claim 11,
The path from the starting point to the ending point includes a plurality of nodes,
The step of repeatedly executing the function is a step of repeatedly executing a function aimed at maximizing an expected reward when the ship moves from the s-th node to the s+1-th node (s is a natural number greater than or equal to 1). doing,
A method for reinforcement learning of an algorithm for generating a ship's path.
According to claim 12,
The function is repeatedly executed for the plurality of virtual paths,
Characterized in that, in each iterative execution step, variables are applied differently to maximize the expected value of the reward.
A method for reinforcement learning of an algorithm for generating a ship's path.
According to claim 13,
As a result of repeatedly executing the function for the plurality of virtual paths,
selecting virtual routes having a calculated expected reward value greater than or equal to a reference value and converting them into a database;
Including more,
A method for reinforcement learning of an algorithm for generating a ship's path.
In the calculation device for calculating the estimated time of arrival of the ship,
an information collection unit that collects location information about the vessel or information related to dynamic environment variables;
a calculation unit for calculating a path from a start point to a destination of the ship by referring to at least one dynamic environment variable or a ship operation database;
a determination unit which determines one of the plurality of paths calculated by the calculation unit as a final path;
a dynamic environment database in which information related to dynamic environment variables is stored;
a control unit controlling the information collection unit, the calculation unit, the determination unit, and the dynamic environment database;
including,
computing device.
According to claim 15,
Further comprising a ship operation database for storing past ship operation information,
computing device.
According to claim 16,
The calculation unit,
Creating a first route from the starting point to the destination by referring to at least one dynamic environment variable, and generating a second route by at least partially modifying the first route by referring to the ship operation database ,
computing device.
According to claim 16,
The ship operation database accumulates and stores result values obtained by reinforcement learning of past ship operation information,
The result values are a function that aims to maximize the reward expected when the ship moves from the s-th node to the s+1-th node (s is a natural number greater than or equal to 1) in an arbitrary path composed of a plurality of nodes. Characterized in that the result values calculated by the iterative operation of
computing device.
According to claim 18,
The function

defined as,
s is the node state, a is the action, γ is the discount rate, Q(s,a) is the action value function for calculating the expected value of the reward to be received when action a is taken in a specific node state s, and a' is the next node state ( Characterized in that it is an action that maximizes the value of the action value function in s'),
computing device.
According to claim 19,
The action value function is

defined as,
s is node state, a is action, E is expected value, R is reward, γ is discount rate, t is time,
Characterized in that Q _π (s, a) is a function for calculating the expected value of the reward to be received when action a is taken in a specific state s,
computing device.

Images