CN112752308B - Mobile prediction wireless edge caching method based on deep reinforcement learning - Google Patents
Mobile prediction wireless edge caching method based on deep reinforcement learning Download PDFInfo
- Publication number
- CN112752308B CN112752308B CN202011620501.1A CN202011620501A CN112752308B CN 112752308 B CN112752308 B CN 112752308B CN 202011620501 A CN202011620501 A CN 202011620501A CN 112752308 B CN112752308 B CN 112752308B
- Authority
- CN
- China
- Prior art keywords
- user
- cache
- service node
- neural network
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/10—Flow control between communication endpoints
- H04W28/14—Flow control between communication endpoints using intermediate storage
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Mobile Radio Communication Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Telephonic Communication Services (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a mobile prediction wireless edge caching method based on deep reinforcement learning, which comprises the following steps: constructing a wireless intelligent cache network model, which comprises a user set, a service node set, a user request content set, a cache content set, a source content library, a user historical track vector and a user classification group; constructing a long-short term memory network model, taking the historical track vector of the user as a prediction position of a prediction user in the next time slot, and classifying to obtain a user classification group; establishing a replacement cache strategy, acquiring a predicted user set of each service node according to the user classification group, and replacing the cache content of the current service node; and constructing a neural network combining Q learning and DQN reinforcement learning, training the neural network to obtain a trained dynamic cache replacement model, and utilizing the dynamic cache replacement model in a cache replacement strategy.
Description
Technical Field
The invention relates to a mobile prediction wireless edge caching method based on deep reinforcement learning, and belongs to the technical field of wireless communication and computers.
Background
With the exponential growth of mobile wireless communication, data demand, and the continuous improvement of device storage and computing capabilities, real-time multimedia services are gradually becoming a major business in 5G communication networks, and human life and work are gradually migrating towards the overall mobile internet, pushing various network functions to the edge of the network, such as edge computing and edge caching. By pre-storing popular content requested by a user, edge caching aims to reduce traffic load and duplicate transmissions in the backhaul network, thereby significantly reducing latency, and therefore, accurately predicting a user's future needs is critical for edge caching replacement. To capture content popularity and the dynamics of a time-varying wireless environment, a policy control framework is introduced into the field of wireless caching. The deep reinforcement learning combines the deep neural network and the Q learning, shows excellent performance in the aspect of solving the complex control problem, and gets more and more attention in the research of wireless edge cache.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a mobile prediction wireless edge caching method based on deep reinforcement learning, which can predict the position of a mobile user by using a long-term and short-term memory network, overcome the influence of the mobility of the user on the cache hit rate, and solve the caching problem in a wireless network by using a neural network framework combining Q learning and reinforcement learning to carry out a cache replacement strategy of a service node, thereby improving the capability of mobile prediction wireless edge caching.
The technical scheme of the invention is as follows:
a mobile prediction wireless edge caching method based on deep reinforcement learning comprises the following steps:
constructing a wireless intelligent cache network model, which comprises a service node model and a service node control model, wherein the service node model comprises a user set, a service node set, a user request content set, a cache content set and a source content library; the service node control model comprises a user historical track vector and a user classification group;
mobile prediction, namely constructing a long-term and short-term memory network model, taking the historical track vector of the user as input, and outputting the predicted position of the user in the next time slot; classifying according to the predicted position of each user in the user set in the next time slot to obtain the user classification group;
establishing a replacement cache strategy, acquiring a predicted user set of each service node in a service node set in the next time slot according to a user classification group, and acquiring replacement contents from a source content library according to history request contents of users in the predicted user set and cache contents of a current service node to replace the cache contents of the current service node;
and (3) constructing a deep learning neural network combined with Q learning, taking a sample state in a state space formed by a prediction user set, a user request content set and a cache content set as an input, taking a certain action in an action space formed by replacement content as an output, training the neural network to obtain a trained dynamic cache replacement model, and utilizing the dynamic cache replacement model in a cache replacement strategy.
Furthermore, the wireless intelligent cache network model operates in a time discrete mode, and in each time slot, the user request content and the user historical track are updated.
Further, the user historical track vector is a position sequence and represents the moving track of the user within a period of time, and the historical track vector of each user is stored in the service node control model;
and inputting the historical track vector of the user into the long-short term memory network model, introducing a weight matrix, and outputting the predicted position of each user in the next time slot.
Further, in the process of training the neural network, a reward function is constructed based on the cache hit rate to train the neural network, and the method specifically comprises the following steps:
constructing a reward function which calculates an instant reward value through an input sample state and an output action and provides the instant reward value to a neural network;
constructing a cache hit rate calculation formula, wherein the cache hit rate refers to the probability that the request content of each user in a user set corresponding to a service node can be found in the cache content of the corresponding service node;
presetting a threshold value, wherein the threshold value belongs to (0,1), acquiring the state of the sample in the next time slot according to the input sample state and the output action, calculating the cache hit rate of the sample in the state of the next time slot according to the cache hit rate calculation formula, comparing with the threshold value, and obtaining a positive instantaneous reward value when the cache hit rate of the sample in the state of the next time slot is greater than the threshold value.
Furthermore, an experience replay mechanism is arranged in the neural network, and the input sample state, the output action, the instant reward value and the state of the sample in the next time slot are combined and stored in an experience replay library to be used as a training sample of the neural network.
Further, the step of constructing the neural network combining Q learning and DQN reinforcement learning specifically includes:
defining an action value function for calculating a Q value through training samples in an experience replay library through Q learning;
the DQN reinforcement learning adopts a neural network to predict a q value, and for each training sample in an experience playback library, the q value of a currently taken action is predicted through the state and the action of the sample, and then the q value of the next state taken action is predicted through the state and the action of the sample in the next time slot;
and constructing a loss function taking the difference value between the q value of the action taken in the next state and the q value of the current action taken as a reference, and iteratively updating the weight parameters of the neural network by using a gradient descent method to make the neural network converge.
The invention has the following beneficial effects:
1. the invention discloses a mobile prediction wireless edge caching method based on deep reinforcement learning, which can predict the position of a mobile user by utilizing a long-term and short-term memory network, can overcome the influence of the mobility of the user on the cache hit rate, and simultaneously utilizes a neural network framework combining Q learning and reinforcement learning to carry out a cache replacement strategy of a service node, solves the caching problem in a wireless network, thereby improving the capability of mobile prediction wireless edge caching.
2. The invention relates to a mobile prediction wireless edge caching method based on deep reinforcement learning, which is characterized in that a reward function based on cache hit rate is established, and a positive instantaneous reward value is given only when the cache hit rate is greater than a threshold value after cache content is replaced, so that the accuracy of a neural network output result is improved.
3. The invention relates to a mobile prediction wireless edge caching method based on deep reinforcement learning, which is characterized in that a prediction user set of each node is obtained according to the prediction position of a user, so that the user can obtain caching resources in a service node as much as possible, and the time delay is reduced.
4. The invention discloses a mobile prediction wireless edge caching method based on deep reinforcement learning, which is characterized in that the calculation of approximate Q values of a neural network is used, the action of obtaining the maximum Q value in each state is generated through iteration, so that an optimal cache replacement strategy is obtained, the neural network continuously updates iteration parameters through gradient descent, so that a loss function tends to be stable to the minimum value, and the whole network is converged.
Drawings
FIG. 1 is an overall flow chart of an embodiment of the present invention;
FIG. 2 is a diagram of an exemplary wireless intelligent cache network model according to an embodiment of the present invention;
FIG. 3 is a flow chart of a caching policy in an embodiment of the invention;
FIG. 4 is a diagram illustrating an example of different movement modes according to an embodiment of the present invention;
fig. 5 is a comparison example diagram of the calculation results after the scheme of the present embodiment is adopted for different movement modes.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
The first embodiment is as follows:
referring to fig. 1, a method for caching mobile prediction wireless edges based on deep reinforcement learning includes the following steps:
the method comprises the steps of constructing a wireless intelligent cache network model, wherein the wireless intelligent cache network model comprises a service node model and a service node control model, and the service node model comprises a user set U ═ { U ═ 1 ,U 2 ,...,U I B, service node set B ═ B 1 ,B 2 ,...,B J }, user request content setCaching content setsAnd source content library O ═ { O 1 ,O 2 ,...,O K };Indicating the request content of the ith user in the t time slotThe cache content of the jth service node in the tth time slot is represented;
the service node control model comprises a user historical track vector and a user classification group;
mobile prediction, namely constructing a long-term and short-term memory network model, taking the historical track vector of the user as input, and outputting the predicted position of the user in the next time slot; classifying according to the predicted position of each user in the user set in the next time slot to obtain the user classification group;
establishing a replacement cache strategy, and acquiring a predicted user set of each service node in the service node set in the next time slot according to the user classification groupAnd based on the predicted user setFrom a user requesting a setAnd caching content setsThe history request content of the user and the cache content of the current service node are obtained, and when the history request content of the user does not exist in the cache content of the current service node, the history request content of the user is obtained from a source content library O ═ O 1 ,O 2 ,...,O K Get the replacement content to replace the cache content of the current service node, that is, get the replacement contentWill be O ═ O 1 ,O 2 ,...,O K Replacing with new content provided;
optimizing model, constructing deep learning neural network combining Q learning and DQN reinforcement learning, and defining state space asTaking the sample state in the state space as input, the motion space is defined as a (t) ={x 1 ,x 2 ,…,x K And (5) taking each action in the action space of the replacement content as output, training the neural network to obtain a trained dynamic cache replacement model, and using the dynamic cache replacement model in a replacement cache strategy.
The implementation utilizes the long-term and short-term memory network to predict the position of the mobile user, can overcome the influence of the mobility of the user on the cache hit rate, and simultaneously utilizes the neural network framework combined with Q learning and reinforcement learning to carry out the cache replacement strategy of the service node, thereby solving the cache problem in the wireless network and further improving the capability of mobile prediction of wireless edge cache.
Example two:
further, the wireless intelligent cache network model runs in a time discrete mode, T ═ {1,2, …, T }. In each time slot, the location information and the request of the user are updated, i.e.Andis updated if the requested content is cached in the collectionIt will transmit directly to the user; otherwise, the content request and delivery need to be sent from a remote server over the backhaul. For updating the cache contents, the slave servicePredictive user set sent by node controllerWill be provided withAndused as input to a neural network to determine the buffer content of the next time slot, i.e.Some of the content in the store will be replaced by new content provided by the remote server.
Further, the user historical track vector is a position sequence and represents the moving track of the user within a period of time, and the historical track vector of each user is stored in the service node control model; defined as a sequence of positions ofA total of beta historical access records are included.
The position sequence is used as the input of a long-time memory network, and the predicted position of the user in the next time slot is realized, namely:whereinHere W is a different weight matrix.
Further, in the process of training the neural network, a reward function is constructed based on the cache hit rate to train the neural network, and the method specifically comprises the following steps:
constructing a reward function, and combining the cache of the service node of the current user after the service node receives the request content of the user and the predicted user set of the service node controller model at each time slot, namely generating a state s (t) Then, it is regarded asInput of the neural network, the neural network will be according to the state s (t) Selecting a certain action a in the action space (t) When the output is taken, the action is executed according to the reward functionObtaining an instantaneous prize value
Constructing a cache hit rate calculation formula:
wherein the content of the first and second substances,indicating when the user requestsCaching at a current serving nodeWhen the information can be found, the value of the indication function is 1, otherwise, the value is 0, and then the cache hit rate of the jth service node is cached, namely the cache hit rate is applied to all the users currently located in the service nodeThe indicator function is evaluated once and finally normalized to find the percentage hit rate.
Presetting a threshold value zeta, when the threshold value zeta belongs to (0,1), if the cache hit rate is larger than the threshold valueA positive reward is obtainedExperiments show that the cache hit rate of zeta 0.6 is better than other cache hit ratesThe cache hit rate of the value, the purpose of the system is to maximize the cache hit rate of each serving node.
Furthermore, an experience playback mechanism is arranged in the neural network, and after receiving the request content and the predicted user set of the user, the service node combines the cache content of the service node of the current user at each time slot, namely, generates the state s (t) Then, as the input of the neural network, the neural network will be based on the state s (t) Selecting a certain action a in the action space (t) When the output is taken, the system can execute the action according to the reward functionObtaining an instantaneous prize valueAnd enters the next state s (t+1) . The four elements will then be combined into a combinationStoring the training samples into an experience playback library to be used as training samples of the neural network.
Further, the step of constructing the neural network combining Q learning and DQN reinforcement learning specifically includes:
defining a motion cost function for computing Q values through training samples in an empirical replay library by Q learning (Q-learning):
where γ ∈ (0,1) represents the discount factor.
Because the large dimension of the action space can consume a large amount of memory, the DQN reinforcement learning adopts a neural network to estimate the q value q π (s (t) ,a (t) ,ω)≈q π (s (t) ,a (t) ) When samples are extracted from an empirical replay library for training, each sample can be calculated in an estimation network under the current sample stateQ value q for taking the action π (s (t) ,a (t) ω) while using the next state s in the sample (t+1) Inputting the value into a target network (structure and estimation network are consistent, and updating is delayed) to calculate the value of the action taken by the next stateDefining the loss function as the square of the difference between the two
Updating a weight parameter omega of the neural network by using a gradient descent method;
the trained weight parameter omega is stable, and the whole neural network is in a convergence state.
In order to make those skilled in the art further understand the solution proposed in the present embodiment, the following detailed description is made with reference to specific embodiments. The embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given.
Fig. 2 shows a wireless intelligent cache network model.
The model mainly comprises service nodes, a controller, a source server, a cache template and the like, introduces a user cache model under the service nodes, and each service node can download content requested by a user on the source server through a backhaul link, locally caches the content requested by the user and directly serves the user in a cell.
Fig. 3 shows a flow chart of the cache policy.
In time slot t, the request and location made by the user. A request end: if cached at the service node, the user-requested content is sent directly to the user and if not downloaded from a remote server (source content repository). Position end: and updating the historical track vector of the user, predicting the mobility of the user through a long-term and short-term memory network, then obtaining a predicted user set corresponding to the service node through a classification function, and finally updating the cache content through a neural network.
As shown in fig. 4, which is an exemplary diagram of different movement scenarios.
In order to study the DRL-based caching scheme proposed in various mobility scenarios, three different mobility patterns were tested and compared, and (a) in fig. 4 is linear mobility: for simulating a user's straight line movement on a street or road. In fig. 4, (b) is a circular motion: this is a typical deterministic motion pattern used to simulate a fixed path trajectory. In fig. 4, (c) is random shift: for simulating irregular movement of the user in an open area.
The calculation result is shown in fig. 5, and the result shows that the algorithm adopting the mobility prediction is superior to the algorithm without the mobility prediction, and the performance gains of the cache hit rate are 14.5%, 19.3% and 10.0% respectively under the conditions of linear, circular and random motions, which indicates that the accurate prediction user plays a key role in content replacement to adapt to the data request of the user.
The above analysis shows that the scheme provided by the invention can obtain better caching capacity than the existing method, and can well improve the caching problem of the user.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (5)
1. A mobile prediction wireless edge caching method based on deep reinforcement learning is characterized by comprising the following steps:
constructing a wireless intelligent cache network model, which comprises a service node model and a service node control model, wherein the service node model comprises a user set, a service node set, a user request content set, a cache content set and a source content library; the service node control model comprises a user historical track vector and a user classification group;
mobile prediction, namely constructing a long-term and short-term memory network model, taking the historical track vector of the user as input, and outputting the predicted position of the user in the next time slot; classifying according to the predicted position of each user in the user set in the next time slot to obtain the user classification group;
establishing a replacement cache strategy, acquiring a predicted user set of each service node in a service node set in the next time slot according to a user classification group, and acquiring replacement contents from a source content library according to history request contents of users in the predicted user set and cache contents of a current service node to replace the cache contents of the current service node;
the optimization model is used for constructing a neural network combining Q learning and DQN reinforcement learning, taking a sample state in a state space formed by a prediction user set, a user request content set and a cache content set as an input, taking a certain action in an action space formed by replacement content as an output, training the neural network to obtain a trained dynamic cache replacement model, and using the dynamic cache replacement model in a cache replacement strategy; the wireless intelligent cache network model operates in a time discrete mode, and in each time slot, user request content and user historical tracks are updated.
2. The method for caching the mobile prediction wireless edge based on the deep reinforcement learning of claim 1, wherein: the user historical track vector is a position sequence and represents the moving track of the user within a period of time, and the historical track vector of each user is stored in the service node control model;
and inputting the historical track vector of the user into the long-short term memory network model, introducing a weight matrix, and outputting the predicted position of each user in the next time slot.
3. The method for mobile prediction wireless edge caching based on deep reinforcement learning according to claim 1, wherein in the process of training the neural network, a reward function is constructed based on a cache hit rate to train the neural network, and the method comprises the following specific steps:
constructing a reward function which calculates an instant reward value through an input sample state and an output action and provides the instant reward value to a neural network;
constructing a cache hit rate calculation formula, wherein the cache hit rate refers to the probability that the request content of each user in a user set corresponding to a service node can be found in the cache content of the corresponding service node;
presetting a threshold value, wherein the threshold value belongs to (0,1), acquiring the state of the sample in the next time slot according to the input sample state and the output action, calculating the cache hit rate of the sample in the state of the next time slot according to the cache hit rate calculation formula, comparing with the threshold value, and obtaining a positive instantaneous reward value when the cache hit rate of the sample in the state of the next time slot is greater than the threshold value.
4. The method of claim 3, wherein the method comprises: the neural network is provided with an experience replay mechanism, and the input sample state, the output action, the instant reward value and the state of the sample in the next time slot are combined and stored in an experience replay library to be used as a training sample of the neural network.
5. The method for mobile prediction wireless edge caching based on deep reinforcement learning of claim 4, wherein the step of constructing the neural network combining Q learning and DQN reinforcement learning specifically comprises:
defining an action value function for calculating a Q value through training samples in an experience replay library through Q learning;
the DQN reinforcement learning adopts a neural network to predict a q value, and for each training sample in an experience playback library, the q value of a currently taken action is predicted through the state and the action of the sample, and then the q value of the next state taken action is predicted through the state and the action of the sample in the next time slot; and constructing a loss function taking the difference value between the q value of the action taken in the next state and the q value of the current action taken as a reference, and iteratively updating the weight parameters of the neural network by using a gradient descent method to make the neural network converge.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011620501.1A CN112752308B (en) | 2020-12-31 | 2020-12-31 | Mobile prediction wireless edge caching method based on deep reinforcement learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011620501.1A CN112752308B (en) | 2020-12-31 | 2020-12-31 | Mobile prediction wireless edge caching method based on deep reinforcement learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112752308A CN112752308A (en) | 2021-05-04 |
CN112752308B true CN112752308B (en) | 2022-08-05 |
Family
ID=75650307
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011620501.1A Active CN112752308B (en) | 2020-12-31 | 2020-12-31 | Mobile prediction wireless edge caching method based on deep reinforcement learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112752308B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113315978B (en) * | 2021-05-13 | 2022-03-15 | 江南大学 | Collaborative online video edge caching method based on federal learning |
CN113422801B (en) * | 2021-05-13 | 2022-12-06 | 河南师范大学 | Edge network node content distribution method, system, device and computer equipment |
CN114025017B (en) * | 2021-11-01 | 2024-04-16 | 杭州电子科技大学 | Network edge caching method, device and equipment based on deep circulation reinforcement learning |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103282891A (en) * | 2010-08-16 | 2013-09-04 | 甲骨文国际公司 | System and method for effective caching using neural networks |
US9047225B1 (en) * | 2012-09-27 | 2015-06-02 | Emc Corporation | Dynamic selection of data replacement protocol for cache |
CN110651279A (en) * | 2017-06-28 | 2020-01-03 | 渊慧科技有限公司 | Training motion selection neural networks with apprentices |
CN111901392A (en) * | 2020-07-06 | 2020-11-06 | 北京邮电大学 | Mobile edge computing-oriented content deployment and distribution method and system |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10595037B2 (en) * | 2016-10-28 | 2020-03-17 | Nec Corporation | Dynamic scene prediction with multiple interacting agents |
EP3850548A1 (en) * | 2018-09-11 | 2021-07-21 | NVIDIA Corporation | Future object trajectory predictions for autonomous machine applications |
US20200134445A1 (en) * | 2018-10-31 | 2020-04-30 | Advanced Micro Devices, Inc. | Architecture for deep q learning |
CN109976909B (en) * | 2019-03-18 | 2022-11-08 | 中南大学 | Learning-based low-delay task scheduling method in edge computing network |
US11636393B2 (en) * | 2019-05-07 | 2023-04-25 | Cerebri AI Inc. | Predictive, machine-learning, time-series computer models suitable for sparse training sets |
EP3748455B1 (en) * | 2019-06-07 | 2022-03-16 | Tata Consultancy Services Limited | A method and a system for hierarchical network based diverse trajectory proposal |
-
2020
- 2020-12-31 CN CN202011620501.1A patent/CN112752308B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103282891A (en) * | 2010-08-16 | 2013-09-04 | 甲骨文国际公司 | System and method for effective caching using neural networks |
US9047225B1 (en) * | 2012-09-27 | 2015-06-02 | Emc Corporation | Dynamic selection of data replacement protocol for cache |
CN110651279A (en) * | 2017-06-28 | 2020-01-03 | 渊慧科技有限公司 | Training motion selection neural networks with apprentices |
CN111901392A (en) * | 2020-07-06 | 2020-11-06 | 北京邮电大学 | Mobile edge computing-oriented content deployment and distribution method and system |
Non-Patent Citations (3)
Title |
---|
基于Bi-LSTM的无人机轨迹预测模型及仿真;杨任农等;《航空工程进展》;20200228(第01期);全文 * |
基于深度学习的智能移动边缘网络缓存;宋旭鸣等;《中国科学院大学学报》;20200115(第01期);全文 * |
基于长短记忆型卷积神经网络的犯罪地理位置预测方法;肖延辉等;《数据分析与知识发现》;20181025(第10期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN112752308A (en) | 2021-05-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112752308B (en) | Mobile prediction wireless edge caching method based on deep reinforcement learning | |
Zhu et al. | Caching transient data for Internet of Things: A deep reinforcement learning approach | |
CN109660598B (en) | Cache replacement method and system for transient data of Internet of things | |
Li et al. | Energy-latency tradeoffs for edge caching and dynamic service migration based on DQN in mobile edge computing | |
Zhang et al. | Toward edge-assisted video content intelligent caching with long short-term memory learning | |
CN109639760A (en) | It is a kind of based on deeply study D2D network in cache policy method | |
CN109982104B (en) | Motion-aware video prefetching and cache replacement decision method in motion edge calculation | |
CN111491331B (en) | Network perception self-adaptive caching method based on transfer learning in fog computing network | |
CN114553963B (en) | Multi-edge node collaborative caching method based on deep neural network in mobile edge calculation | |
CN115809147B (en) | Multi-edge collaborative cache scheduling optimization method, system and model training method | |
Yan et al. | Distributed edge caching with content recommendation in fog-rans via deep reinforcement learning | |
Wang et al. | Deepchunk: Deep q-learning for chunk-based caching in wireless data processing networks | |
Negara et al. | Caching and machine learning integration methods on named data network: A survey | |
CN113687960A (en) | Edge calculation intelligent caching method based on deep reinforcement learning | |
Malektaji et al. | Deep reinforcement learning-based content migration for edge content delivery networks with vehicular nodes | |
Li et al. | DQN-enabled content caching and quantum ant colony-based computation offloading in MEC | |
CN117221403A (en) | Content caching method based on user movement and federal caching decision | |
CN116321307A (en) | Bidirectional cache placement method based on deep reinforcement learning in non-cellular network | |
Yu et al. | Mobility-aware proactive edge caching for large files in the internet of vehicles | |
Liu et al. | Mobility-aware video prefetch caching and replacement strategies in mobile-edge computing networks | |
CN113114762B (en) | Data caching method and system | |
CN114786200A (en) | Intelligent data caching method based on cooperative sensing | |
CN115129888A (en) | Active content caching method based on network edge knowledge graph | |
Hatami et al. | Online caching policy with user preferences and time-dependent requests: A reinforcement learning approach | |
Li et al. | A deep reinforcement learning-based content updating algorithm for high definition map edge caching |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |