CN114143212B - Social learning method for smart city - Google Patents
Social learning method for smart city Download PDFInfo
- Publication number
- CN114143212B CN114143212B CN202111425250.6A CN202111425250A CN114143212B CN 114143212 B CN114143212 B CN 114143212B CN 202111425250 A CN202111425250 A CN 202111425250A CN 114143212 B CN114143212 B CN 114143212B
- Authority
- CN
- China
- Prior art keywords
- task
- evaluation model
- edge server
- decision
- internet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000009326 social learning Effects 0.000 title claims abstract description 26
- 238000013210 evaluation model Methods 0.000 claims abstract description 87
- 230000013016 learning Effects 0.000 claims abstract description 35
- 230000002776 aggregation Effects 0.000 claims abstract description 22
- 238000004220 aggregation Methods 0.000 claims abstract description 22
- 238000013526 transfer learning Methods 0.000 claims abstract description 15
- 230000002787 reinforcement Effects 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims description 99
- 238000005457 optimization Methods 0.000 claims description 16
- 238000005265 energy consumption Methods 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 9
- 230000005012 migration Effects 0.000 abstract description 2
- 238000013508 migration Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000001228 spectrum Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013468 resource allocation Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007786 learning performance Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 230000001012 protector Effects 0.000 description 1
- 230000003997 social interaction Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y20/00—Information sensed or collected by the things
- G16Y20/30—Information sensed or collected by the things relating to resources, e.g. consumed power
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Tourism & Hospitality (AREA)
- Computer Networks & Wireless Communication (AREA)
- Human Resources & Organizations (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Educational Administration (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Development Economics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a social learning method for a smart city, which comprises the following steps: constructing a layered social learning system; establishing a task evaluation model based on deep reinforcement learning, and optimizing the task evaluation model by using the task states and channel states of all Internet of things equipment to obtain a basic decision; the edge server utilizes the task evaluation model received by the federal learning edge aggregation and optimizes the task evaluation model on the edge server according to the basic decision to obtain a high-level decision; the edge server guides a model in the Internet of things equipment by using transfer learning; the cloud server utilizes the task evaluation model received by the federal learning cloud aggregation to make a city-level decision according to the high-level decision and the task evaluation model on the cloud server, and utilizes the migration learning to guide the task evaluation model on the edge server. According to the method, the cooperation among the intelligent agents in the layer is improved by using federal learning, and the guidance of the upper layer to the lower layer is realized by using transfer learning among the layers, so that the performance of the model is improved.
Description
Technical Field
The invention belongs to the technical field of big data, and particularly relates to a social learning method for a smart city.
Background
The recent worsening trends of population explosion, resource imbalance, traffic jam and the like in the urbanization process provide increasingly high requirements for high-quality life of citizens. With the unprecedented proliferation of 5G, internet of things (IoT) and Artificial Intelligence (AI), the smart city becomes a new trend of city development. With the popularization of smart cities, the data volume from internet of things equipment will increase sharply in 2021, and 850Zettabytes will be reached. Billions of internet of things devices are associated with smart cities, building various smart small areas for the entire city. These internet of things devices are typically deployed with moderate computing capabilities, such as smart street lights, smart traffic lights, smart surveillance cameras, and smart phones. In addition, the communication function bridges the gap between the Internet of things equipment, the user and even the whole city, and blood vessels are provided for the smart city. Some resource factors, such as underutilized spectrum resources, huge bandwidth costs and limited computing power, outweigh the benefits of the smart city, i.e., the gradual depletion and crowding of cells and vessels in the smart city.
To free the potential of smart cities, there are many research trends that can address the challenges described above. For example, the cognitive internet of things enables internet of things equipment to flexibly sense and dynamically access the frequency spectrum, so that the frequency spectrum requirement of a smart city is relieved. Edge computing and fog computing push computing tasks and services from the cloud server to the network edge, further reducing bandwidth consumption. There are still many problems to be solved. 1) Due to strict requirements of smart cities on delay, decisions on spectrum access and calculation allocation need to be made in advance and have high precision, and the requirements urge a great deal of research on artificial intelligence benefit strategies; 2) the traditional artificial intelligence method for constructing an intelligent decision usually depends on providing mass data and training on one or more cloud servers, and the problems further aggravate the problems of bandwidth cost, time efficiency and the like; 3) edge intelligence involves pushing learning intelligence from one or several cloud servers to the network edge, but they ignore the cooperative nature between edge servers, resulting in inefficient learning resources and even degraded learning performance; 4) behind the operation of the smart city, an obvious social level system exists, and the hierarchical structure consists of the internet of things equipment, an edge server determining the operation of the internet of things equipment and a cloud server determining the operation of the edge server. Existing edge intelligence also ignores social decisions of smart cities.
Disclosure of Invention
Aiming at the technical problems, inspired by an operation mechanism of efficient cooperation of human society and social interaction between people, the invention provides a social learning method facing a smart city, which is a social, learning-based and cognition scheme, and establishes grades according to the characteristics of an agent, makes interrelated decisions, enables information to flow interactively, and can meet various requirements of the smart city, such as reasonable resource allocation. In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a social learning method for smart cities comprises the following steps:
s1, constructing a layered social learning system comprising a cloud server, an edge server and Internet of things equipment, wherein the Internet of things equipment is connected with the edge server through a wireless network, and the edge server is connected with the cloud server;
s2, respectively establishing task evaluation models in the Internet of things equipment, the edge server and the cloud server based on deep reinforcement learning;
s3, the Internet of things equipment optimizes the task evaluation model by using the task states and channel states obtained by all the Internet of things equipment, obtains a basic task processing decision according to the optimized task evaluation model, and sends the basic task processing decision and the optimized task evaluation model to the corresponding edge server;
s4, the edge server performs edge aggregation on the task evaluation model sent by the Internet of things equipment by means of federal learning, optimizes the task evaluation model on the edge server according to the basic task processing decision sent by the Internet of things equipment to obtain a task processing high-level decision, and sends the task processing high-level decision to the Internet of things equipment;
s5, the edge server guides a task evaluation model in the Internet of things equipment by using transfer learning, and sends the task evaluation model after task processing high-level decision and optimization to the cloud server;
and S6, the cloud server carries out cloud aggregation on the task evaluation model sent by the edge server by means of federal learning, makes a task processing city-level decision according to the task processing high-level decision sent by the edge server and the task evaluation model on the cloud server, sends the task processing city-level decision to the edge server, and guides the task evaluation model in the edge server by means of transfer learning.
In step S3, the task state includes a CPU cycle and a task data volume of the task, and the channel state includes a grant channel gain, an unlicensed channel gain, and a channel occupancy state of the wireless network.
The equipment of the Internet of things comprises a primary user and a secondary user, wherein the primary user is connected with an edge server through an authorized channel, the secondary user is connected with the edge server through an authorized channel or an unauthorized channel, and when the secondary user uses the authorized channel, the connection of the primary user is not influenced.
The step S3 includes the steps of:
s3.1, each piece of Internet of things equipment acquires a task state and a channel state in the current environment and obtains a task processing preliminary decision according to a corresponding task evaluation model;
s3.2, each piece of Internet of things equipment integrates task processing preliminary decisions of all pieces of Internet of things equipment, and a task evaluation model is optimized once by taking the weighted sum of the minimized total processing delay and the energy consumption as a target;
and S3.3, the Internet of things equipment makes a basic task processing decision according to the once optimized task evaluation model, and sends the basic task processing decision and the once optimized task evaluation model to the corresponding edge server.
The task processing basic decision and the task processing preliminary decision comprise an unloading decision, channel selection of a wireless network and computing resource budget.
The step S4 includes the following steps:
s4.1, each edge server acquires a channel state and edge available computing resources in the current environment, performs edge aggregation on the received once-optimized task evaluation model by using federal learning, and sends the task evaluation model subjected to edge aggregation to corresponding Internet of things equipment;
s4.2, each edge server obtains a task processing high-level preliminary decision according to the received task processing basic decision, the channel state and the edge available computing resources obtained in the step S4.1 and a task evaluation model on the edge server;
s4.3, each edge server integrates the task processing high-level preliminary decisions of all the edge servers, and performs primary optimization on the task evaluation model on the edge server by taking cost minimization as a target, and obtains the task processing high-level decisions according to the task evaluation model after primary optimization;
and S4.4, the edge server sends the task processing high-level decision to the corresponding Internet of things equipment, and the Internet of things equipment carries out secondary optimization on the task evaluation model on the Internet of things equipment by using the task processing high-level decision.
The task processing high-level decision and the task processing high-level preliminary decision comprise evaluation results of basic decisions, cooperative edge server selection and edge computing resource contribution.
The cost is equal to the difference between the edge computing resource contribution minus the cost of cooperating with other edge servers.
The step S6 includes the following steps:
s6.1, the cloud server obtains a channel state and cloud available computing resources in the current environment, performs cloud aggregation on the received task evaluation model sent by the edge server by using federal learning, and sends the task evaluation model after cloud aggregation to the edge server;
s6.2, the cloud server inputs the received task processing high-level decision, the channel state obtained in the step S5.1 and the cloud available computing resources into a task evaluation model on the cloud server to obtain a task processing city-level decision, sends the task processing city-level decision to a corresponding edge server, and guides the task evaluation model in the edge server by using transfer learning;
and S6.3, performing secondary optimization on the task evaluation model on the edge server by the edge server according to the task processing city-level decision.
The task processing city level decision comprises an evaluation result of a high-level decision and the contribution amount of cloud computing resources.
The invention has the beneficial effects that:
according to the method, a layered social learning system is constructed according to different computing capacities of the agents, the agents in the layer improve the cooperation capacity among the agents in the layer through federal learning, and the upper layer guides the lower layer through transfer learning among the layers, so that the performance of the model is improved; all the agents cooperate with each other, and a single agent considers the decisions of other agents when making decisions, so that the transmission delay, the energy consumption and the bandwidth utilization rate can be jointly optimized when making decisions on task unloading, spectrum selection and calculation resource allocation.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a basic architecture diagram of the present invention.
Fig. 2 shows the relationship among the cloud server, the edge server, and the internet of things device.
FIG. 3 is a flow chart of the present invention.
Fig. 4 is a graph showing the comparison effect between the random offloading scheme and the local offloading scheme between the average SINR, the average delay, and the average energy consumption.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
With the rapid development of machine intelligence, it can be considered that many machines constitute a machine society like humans. For human society, social learning theories point out the limiting effect of social features on human behavior. The operational mechanisms of human society have incentives for individuals and promise a number of benefits regarding socialization features between individuals. People in the society are divided into different levels according to a certain standard, and the social relationship in the society enables the human society to be more harmonious and efficient. For example, in terms of decision making, a high level person need not collect all data from a low level person. Higher-level people are more concerned about learning decisions from lower-level people, helping them make higher-level decisions. That is, in real society, the president does not need to know how a state owner decides, and he only needs to learn the decision results. Inspired by the efficient collaborative operation mechanism of human society, the invention introduces the efficient collaborative operation mechanism into the machine society and provides social learning to solve various requirements in smart cities, such as reasonable allocation of resources. Unlike other enabling technologies of edge intelligence, such as federal learning, the scheme has more human-social characteristics, such as social relations: collaboration and competition. Competition and collaboration, as representatives of social relationships, are all components of social rules. The interactivity, the enthusiasm and the learning efficiency of individuals can be improved through competition. In addition, collaboration can expose private knowledge to enhance the ability and utility of all individuals.
A social learning method for smart cities, as shown in fig. 1-4, comprising the following steps:
s1, constructing a layered social learning system comprising a cloud server, an edge server and Internet of things equipment, wherein the Internet of things equipment is connected with the edge server through a wireless network, and the edge server is connected with the cloud server;
the cloud server, the edge server and the Internet of things equipment are all intelligent agents, and due to the fact that resource factors such as computing capacity, sensing range and communication mode of the intelligent agents are different, the cloud server, the edge server and the Internet of things equipment with different resources are respectively arranged on different layers in the layered social learning system, the cloud server on the high layer has strong data analysis and computing capacity, ubiquitous, convenient and on-demand network access is supported, and high communication overhead is caused in a long distance. The large-scale internet of things equipment at the bottom layer is provided with a plurality of time-sensitive and calculation-intensive tasks which need processing, but the large-scale internet of things equipment has limited sensing capability and is limited by the calculation capability. The middle layer is an edge server layer which is closer to the bottom layer Internet of things equipment than the cloud server and can provide medium computing capacity; through the wireless network connected with the Internet of things equipment and the backbone network connected with the cloud server, the edge server can relieve the barrier between the highest layer and the lowest layer through coordinating and integrating various types of resources.
The channels of the wireless network comprise an authorized channel and an unauthorized channel, the internet of things equipment comprises a primary user and a secondary user, the primary user mainly uses the authorized channel to transmit signals, the secondary user mainly uses the unauthorized channel to transmit signals, and when the secondary user wants to use the authorized channel of the primary user to unload, wireless transmission of the primary user needs to be guaranteed not to be affected.
S2, respectively establishing task evaluation models in the Internet of things equipment, the edge server and the cloud server based on Deep Reinforcement Learning (DRL);
s3, the IOT equipment optimizes the task evaluation model on the IOT equipment by using the task states and channel states obtained by all the IOT equipment, obtains the basic task processing decision according to the optimized task evaluation model, and sends the basic task processing decision and the optimized task evaluation model to the corresponding edge server, and the method comprises the following steps:
s3.1, each piece of Internet of things equipment acquires a task state and a channel state in the current environment, and obtains a task processing preliminary decision according to a task evaluation model on the piece of Internet of things equipment;
the task state comprises a CPU period and a task data volume of a task, and the channel state comprises an authorized channel gain, an unauthorized channel gain and a channel occupation state;
s3.2, each piece of Internet of things equipment integrates task processing preliminary decisions of all pieces of Internet of things equipment, and a task evaluation model on the piece of Internet of things equipment is optimized for the first time by taking the weighted sum of the minimized total processing delay and the energy consumption as a target;
when making a decision, a single Internet of things device considers the decision of other Internet of things devices at the same time, and cooperation among agents can be realized.
S3.3, the Internet of things equipment makes a basic task processing decision according to the once optimized task evaluation model and sends the basic task processing decision and the once optimized task evaluation model to the connected edge server;
the task processing basic decision and the task processing preliminary decision comprise unloading decision, channel selection and computing resource budget, wherein the unloading decision refers to local processing or remote unloading of the task, and the computing resource budget refers to evaluation of computing resources required by task execution.
S4, the edge server conducts edge aggregation on the task evaluation model sent by the Internet of things equipment through federal learning, optimizes the task evaluation model on the edge server according to the basic decision of the Internet of things equipment, obtains a high-level decision according to the optimized task evaluation model, and sends the high-level decision and the optimized task evaluation model to the cloud server, and the method comprises the following steps:
s4.1, each edge server acquires a channel state and edge available computing resources in the current environment, edge aggregation is carried out on task evaluation models sent by the connected Internet of things equipment by using federal learning, and the task evaluation models after edge aggregation are sent to the corresponding Internet of things equipment;
the edge server can acquire a more accurate channel state because the edge server has a stronger computing power. By means of federal learning, efficient machine learning can be performed among devices on the same layer on the premise that information safety, data privacy and legality in the data exchange process are guaranteed.
S4.2, each edge server obtains a task processing high-level preliminary decision according to the received task processing basic decision, the channel state obtained in the step S4.1, the edge available computing resources and a task evaluation model on the edge server;
s4.3, each edge server integrates the task processing high-level preliminary decisions of all the edge servers, and performs primary optimization on the task evaluation model on the edge server by taking cost minimization as a target, and obtains the task processing high-level decisions according to the task evaluation model after primary optimization;
the cost is calculated by subtracting the cost of cooperating with other edge servers from the edge computing resource contribution.
S4.4, the edge server sends the task processing high-level decision to corresponding Internet of things equipment, and the Internet of things equipment carries out secondary optimization on a task evaluation model on the Internet of things equipment by using the task processing high-level decision;
the task processing high-level decision and the task processing high-level preliminary decision comprise an evaluation result of a basic decision, cooperative edge server selection and edge computing resource contribution, the evaluation result of the basic decision is whether the Internet of things equipment is approved to execute the unloading decision, the cooperative edge server selection is that an edge server set which needs cooperation during remote task unloading is also to be unloaded to which edge servers to process the task, and the edge computing resource contribution is computing resources distributed by each edge server according to respective available computing resources during remote task unloading. Since the computational resources of each edge server are limited, a reasonable distribution of computational resources can be achieved through cooperation between the edge servers.
S5, the edge server guides a task evaluation model in the Internet of things equipment by using transfer learning, and sends the task evaluation model after task processing high-level decision and primary optimization of the edge server to the cloud server;
the transfer learning is used as a cross-layer learning method, so that the vertical flow of data can be realized, the training process is accelerated, the learning efficiency of the training of a bottom layer model is optimized, and the resource utilization rate is improved. Or the user does not need to learn from the beginning like most training processes, so that the computing resources are greatly saved, and the time efficiency is high. The transfer learning in the embodiment is inspired by a 'teacher-student' structure, which is a special cooperation in social relations, namely, cooperation between teachers and students. The teacher shares the learned knowledge with the students, thereby realizing the flow of the knowledge, improving the abilities of the students and obviously reducing the time and resources consumed by the students for learning the related knowledge.
S6, the cloud server carries out cloud aggregation on the task evaluation model sent by the edge server by means of federal learning, makes a task processing city-level decision according to the task processing high-level decision sent by the edge server and the task evaluation model on the cloud server, sends the task processing city-level decision to the edge server, and guides the task evaluation model in the edge server by means of migration learning, so that guidance of the cloud server to the edge server is achieved, and the method comprises the following steps:
s6.1, the cloud server obtains a channel state and cloud available computing resources in the current environment, performs cloud aggregation on the received task evaluation model sent by the edge server by using federal learning, and sends the task evaluation model after cloud aggregation to the edge server;
the cloud server has the strongest computing capability, so that the cloud server can acquire a more accurate channel state.
S6.2, the cloud server inputs the received task processing high-level decision, the channel state obtained in the step S6.1 and the cloud available computing resource into a task evaluation model on the cloud server to obtain a task processing city-level decision, sends the task processing city-level decision to the corresponding edge server, and guides the task evaluation model in the edge server by using transfer learning;
the task processing city-level decision comprises an evaluation result of a high-level decision and a cloud computing resource contribution amount. The evaluation result of the high-level decision is whether the edge server is approved to execute the unloading decision, and the contribution amount of the cloud computing resources is computing resources which need to be contributed by the cloud server when the cloud server executes the remote unloading task. Through the guide of the upper-layer intelligent agent, more information can be transmitted to the lower-layer intelligent agent, the data storage is increased, and the data quality is improved.
And S6.3, performing secondary optimization on the task evaluation model on the edge server by the edge server according to the task processing city-level decision sent by the cloud server.
The process of social learning among three layers of intelligent agents comprises the following steps: the Internet of things equipment serves as eyes and ears and can sense the environment and preprocess sensed data, due to the limitation of the sensing range, the Internet of things equipment can make basic task processing decisions for various tasks on the Internet of things equipment and upload the basic decisions to the edge server, the edge server carries out aggregation processing on the basic decisions transmitted by the Internet of things equipment in different geographic regions and evaluates the basic decisions of the Internet of things equipment according to the sensed state to make high task processing layer decisions, similarly, after the edge server makes the high task processing layer decisions, the high task processing layer decisions are uploaded to the cloud server, the cloud server carries out aggregation processing on the received high task processing layer decisions of all the edge servers in the smart city to make comprehensive task processing city level decisions, and then the upper layer guides the lower layer to make more accurate decisions through transfer learning, meanwhile, each agent can obtain knowledge through autonomous learning to realize the sharing of perception data, and takes action, namely decision through learning the behaviors of other agents at the same layer or higher layers and corresponding feedback.
In this embodiment, the edge server only focuses on the basic task processing decisions made by the internet of things device, regardless of how the internet of things device makes the basic task processing decisions, and similarly, the cloud server also only focuses on the high task processing decisions made by the edge server, regardless of how the edge server makes the high task processing decisions. All models are realized by operating model parameters during transmission, optimization or updating, and the problems that the environment cannot be accurately perceived due to the limitation of the perception capability of the Internet of things equipment, or wrong unloading decisions are made due to data noise or certain uncertainty in perceived data, and even the consumption of communication and computing resources is increased and the time efficiency is reduced are solved. As the wrong offloading decision may result in the loss of offloading opportunities and the unavailability of available spectrum resources, the congestion, energy consumption and cost of the wireless link are increased, and the bandwidth utilization is reduced. The federal learning and the transfer learning are both prior art, and are not described in detail in this embodiment.
This embodiment can use smart power grids and fields such as wisdom traffic in the wisdom city, when using in the smart power grids field, thing networking equipment is equivalent to the monitoring power supply equipment of a certain geographical region like the circuit breaker, the fuse protector, the ampere meter, the camera equipment of voltmeter etc, edge server is equivalent to the local monitoring machine of a certain geographical region, cloud server is equivalent to the total monitoring center of wired connection local monitoring machine, because camera equipment's computing capability is limited can't handle whole image data, can utilize this application to make relevant image processing decision-making in order to carry out reasonable resource distribution between camera equipment, local monitoring machine and the total monitoring center. When the intelligent traffic system is applied to the field of intelligent traffic, the internet of things equipment is equivalent to traffic monitoring equipment in a certain geographic area, such as fixed or movable camera equipment, the edge server is equivalent to Road Side Units (RSUs) in the certain geographic area, and the cloud server is equivalent to a traffic monitoring center connected with the Road Side Units in a wired mode.
Fig. 4 shows the comparison results of the present application, the random offload scheme, and the local offload scheme in terms of Interference Signal to Noise Ratio (SINR), time delay, and power consumption. The local unloading scheme means that the internet of things equipment only executes tasks locally, and the random unloading scheme randomly determines whether to unload the tasks. Fig. 4(a) is a comparison result of the summed average SINR of Primary Users (PUs), Secondary Users (SUs) and all internet of things devices, that is, the Primary Users and the Secondary Users, and it can be seen that the average SINR of the present application is higher than that of the conventional scheme, and the average increase exceeds 22.2%. This is because the social learning scheme proposed by the present invention can determine the optimal offloading and communication strategies to perform the task.
In addition, in order to further evaluate the time efficiency and energy efficiency of the present application, the average delay and average energy consumption are given in fig. 4(b) and 4(c), respectively, and compared with the conventional schemes (including the random offloading scheme and the local offloading scheme), it can be seen that the present invention has lower average delay and average energy consumption. In terms of latency, the average latency of the present application is 60.9% lower than the local offload scheme and 35.7% lower than the random offload scheme. In terms of energy consumption, the average energy consumption of the method is 20% lower than that of a local unloading scheme and 8.2% lower than that of a random unloading scheme. This is because the internet of things devices select the currently optimal offloading and communication strategy to perform the task, the average delay and energy consumption can be reduced. In the traditional scheme, the internet of things equipment randomly selects a strategy or only executes a task locally, so that delay and high energy consumption are caused.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. A social learning method for smart cities is characterized by comprising the following steps:
s1, constructing a hierarchical social learning system comprising a cloud server, an edge server and Internet of things equipment, wherein the Internet of things equipment is connected with the edge server through a wireless network, and the edge server is connected with the cloud server;
s2, respectively establishing task evaluation models in the Internet of things equipment, the edge server and the cloud server based on deep reinforcement learning;
s3, the Internet of things equipment optimizes the task evaluation model by using the task states and channel states obtained by all the Internet of things equipment, obtains a basic task processing decision according to the optimized task evaluation model, and sends the basic task processing decision and the optimized task evaluation model to the corresponding edge server;
the step S3 includes the following steps:
s3.1, each piece of Internet of things equipment acquires a task state and a channel state in the current environment and obtains a task processing preliminary decision according to a corresponding task evaluation model;
s3.2, each piece of Internet of things equipment integrates task processing preliminary decisions of all pieces of Internet of things equipment, and a task evaluation model is optimized once by taking the weighted sum of the minimized total processing delay and the energy consumption as a target;
s3.3, the Internet of things equipment makes a basic task processing decision according to the once optimized task evaluation model and sends the basic task processing decision and the once optimized task evaluation model to a corresponding edge server;
s4, the edge server performs edge aggregation on the task evaluation model sent by the Internet of things equipment by means of federal learning, optimizes the task evaluation model on the edge server according to the basic task processing decision sent by the Internet of things equipment to obtain a task processing high-level decision, and sends the task processing high-level decision to the Internet of things equipment;
s5, the edge server guides a task evaluation model in the Internet of things equipment by using transfer learning, and sends the task evaluation model after task processing high-level decision and optimization to the cloud server;
and S6, the cloud server performs cloud aggregation on the task evaluation model sent by the edge server by means of federal learning, makes a task processing city-level decision according to the task processing high-level decision sent by the edge server and the task evaluation model on the cloud server, sends the task processing city-level decision to the edge server, and guides the task evaluation model in the edge server by means of transfer learning.
2. The wisdom city-oriented social learning method of claim 1, wherein in step S3, the task state includes a CPU cycle of the task and a task data volume, and the channel state includes an authorized channel gain, an unauthorized channel gain, and a channel occupancy state of the wireless network.
3. The social learning method oriented to a smart city as claimed in claim 2, wherein the internet of things device comprises a primary user and a secondary user, the primary user is connected with the edge server through an authorized channel, the secondary user is connected with the edge server through an authorized channel or an unauthorized channel, and when the secondary user uses the authorized channel, the connection of the primary user is not affected.
4. The wisdom city-oriented social learning method of claim 1, wherein the task processing basic decision and the task processing preliminary decision each comprise an offload decision, a channel selection of a wireless network, and a computational resource budget.
5. The social learning method for smart cities as claimed in claim 1, wherein the step S4 comprises the steps of:
s4.1, each edge server acquires a channel state and edge available computing resources in the current environment, performs edge aggregation on the received once-optimized task evaluation model by using federal learning, and sends the task evaluation model subjected to edge aggregation to corresponding Internet of things equipment;
s4.2, each edge server obtains a task processing high-level preliminary decision according to the received task processing basic decision, the channel state and the edge available computing resources obtained in the step S4.1 and a task evaluation model on the edge server;
s4.3, each edge server integrates the task processing high-level preliminary decisions of all the edge servers, and performs primary optimization on the task evaluation model on the edge server by taking cost minimization as a target, and obtains the task processing high-level decisions according to the task evaluation model after primary optimization;
and S4.4, the edge server sends the task processing high-level decision to the corresponding Internet of things equipment, and the Internet of things equipment carries out secondary optimization on the task evaluation model on the Internet of things equipment by using the task processing high-level decision.
6. The wisdom city-oriented social learning method of claim 5, wherein the task processing high-level decision and the task processing high-level preliminary decision each comprise an evaluation result of a basic decision, collaborative edge server selection and an edge computing resource contribution.
7. The wisdom city-oriented social learning method of claim 6, wherein the cost is equal to a difference between an edge computing resource contribution minus a cost of cooperating with other edge servers.
8. The social learning method for smart cities as claimed in claim 1, wherein the step S6 comprises the steps of:
s6.1, the cloud server obtains a channel state and cloud available computing resources in the current environment, carries out cloud aggregation on the received task evaluation model sent by the edge server by means of federal learning, and sends the task evaluation model after cloud aggregation to the edge server;
s6.2, the cloud server inputs the received task processing high-level decision, the channel state obtained in the step S5.1 and the cloud available computing resources into a task evaluation model on the cloud server to obtain a task processing city-level decision, sends the task processing city-level decision to a corresponding edge server, and guides the task evaluation model in the edge server by using transfer learning;
and S6.3, performing secondary optimization on the task evaluation model on the edge server by the edge server according to the task processing city-level decision.
9. The wisdom city-oriented social learning method of claim 8, wherein the task processing city-level decision comprises an evaluation result of a high-level decision and a cloud computing resource contribution amount.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111425250.6A CN114143212B (en) | 2021-11-26 | 2021-11-26 | Social learning method for smart city |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111425250.6A CN114143212B (en) | 2021-11-26 | 2021-11-26 | Social learning method for smart city |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114143212A CN114143212A (en) | 2022-03-04 |
| CN114143212B true CN114143212B (en) | 2022-09-16 |
Family
ID=80388877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111425250.6A Active CN114143212B (en) | 2021-11-26 | 2021-11-26 | Social learning method for smart city |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114143212B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021139537A1 (en) * | 2020-01-08 | 2021-07-15 | 上海交通大学 | Power control and resource allocation based task offloading method in industrial internet of things |
| CN113543074A (en) * | 2021-06-15 | 2021-10-22 | 南京航空航天大学 | Joint computing migration and resource allocation method based on vehicle-road cloud cooperation |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112134916B (en) * | 2020-07-21 | 2021-06-11 | 南京邮电大学 | Cloud edge collaborative computing migration method based on deep reinforcement learning |
| CN112181666B (en) * | 2020-10-26 | 2023-09-01 | 华侨大学 | Equipment assessment and federal learning importance aggregation method based on edge intelligence |
| CN112565331B (en) * | 2020-11-02 | 2022-08-12 | 中山大学 | Edge calculation-based end-edge collaborative federated learning optimization method |
| CN112817653A (en) * | 2021-01-22 | 2021-05-18 | 西安交通大学 | Cloud-side-based federated learning calculation unloading computing system and method |
| CN112988345B (en) * | 2021-02-09 | 2024-04-02 | 江南大学 | Dependency task unloading method and device based on mobile edge calculation |
| CN113408675A (en) * | 2021-08-20 | 2021-09-17 | 深圳市沃易科技有限公司 | Intelligent unloading optimization method and system based on federal learning |
-
2021
- 2021-11-26 CN CN202111425250.6A patent/CN114143212B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021139537A1 (en) * | 2020-01-08 | 2021-07-15 | 上海交通大学 | Power control and resource allocation based task offloading method in industrial internet of things |
| CN113543074A (en) * | 2021-06-15 | 2021-10-22 | 南京航空航天大学 | Joint computing migration and resource allocation method based on vehicle-road cloud cooperation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114143212A (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Intelligent cognitive radio in 5G: AI-based hierarchical cognitive cellular networks | |
| Liu et al. | Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks | |
| Jiang et al. | Resource allocation of video streaming over vehicular networks: A survey, some research issues and challenges | |
| Luo et al. | Artificial intelligence powered mobile networks: From cognition to decision | |
| Huang et al. | Multi-agent deep reinforcement learning for computation offloading and interference coordination in small cell networks | |
| Zhou et al. | When collaboration hugs intelligence: Content delivery over ultra-dense networks | |
| Wang et al. | Mathematical modeling for network selection in heterogeneous wireless networks—A tutorial | |
| Ali et al. | Hybrid fuzzy logic scheme for efficient channel utilization in cognitive radio networks | |
| CN111641681A (en) | Internet of things service unloading decision method based on edge calculation and deep reinforcement learning | |
| Liwang et al. | Game theory based opportunistic computation offloading in cloud-enabled IoV | |
| Sangeetha et al. | An improved congestion-aware routing mechanism in sensor networks using fuzzy rule sets | |
| Wang et al. | Joint resource allocation and power control for D2D communication with deep reinforcement learning in MCC | |
| Zhang et al. | Fuzzy logic-based resource allocation algorithm for V2X communications in 5G cellular networks | |
| Wang et al. | QoS‐enabled resource allocation algorithm in internet of vehicles with mobile edge computing | |
| Shahraki et al. | A comparative node evaluation model for highly heterogeneous massive‐scale Internet of Things‐Mist networks | |
| Zhong et al. | A vertical handoff decision scheme using subjective-objective weighting and grey relational analysis in cognitive heterogeneous networks | |
| Abood et al. | Intelligent network slicing in V2X networks–A comprehensive review | |
| Zhao et al. | Socialized learning for smart cities: Cognitive paradigm, methodology, and solution | |
| Bao et al. | Resource allocation with edge-cloud collaborative traffic prediction in integrated radio and optical networks | |
| Ji et al. | Multi-Agent Reinforcement Learning Resources Allocation Method Using Dueling Double Deep Q-Network in Vehicular Networks | |
| Xue et al. | Collaborative computation offloading and resource allocation based on dynamic pricing in mobile edge computing | |
| Ren et al. | Incentivized social-aware proactive device caching with user preference prediction | |
| CN108882377A (en) | Resource allocation methods in LTE-R based on cognition | |
| CN114143212B (en) | Social learning method for smart city | |
| Qi et al. | Social prediction-based handover in collaborative-edge-computing-enabled vehicular networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |