CN111586091B - Edge computing gateway system for realizing computing power assembly - Google Patents
Edge computing gateway system for realizing computing power assembly Download PDFInfo
- Publication number
- CN111586091B CN111586091B CN202010219671.2A CN202010219671A CN111586091B CN 111586091 B CN111586091 B CN 111586091B CN 202010219671 A CN202010219671 A CN 202010219671A CN 111586091 B CN111586091 B CN 111586091B
- Authority
- CN
- China
- Prior art keywords
- data
- edge computing
- computing
- module
- structured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 claims abstract description 79
- 238000004891 communication Methods 0.000 claims abstract description 38
- 238000013500 data storage Methods 0.000 claims description 25
- 238000004458 analytical method Methods 0.000 claims description 22
- 238000007726 management method Methods 0.000 claims description 22
- 238000011156 evaluation Methods 0.000 claims description 19
- 230000008447 perception Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 description 26
- 238000012806 monitoring device Methods 0.000 description 15
- 238000000034 method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses an edge computing gateway system for realizing computing power assembly, which comprises: the system comprises a data source node, an edge computing gateway and a cloud collaborative computing server; the data source nodes are provided with a plurality of nodes and used for sensing and generating data needing to be calculated and processed; the edge computing gateway is in communication connection with a plurality of data source nodes and is used for acquiring, computing and processing data perceptively generated by the data source nodes and converting a network protocol; the cloud-side collaborative computing server is in communication connection with the edge computing gateway, and is used for collaboratively computing the data generated by the data source node sensing by the edge computing gateway and feeding back the processed result to the edge computing gateway.
Description
Technical Field
The invention relates to the technical field of Internet of things, in particular to an edge computing gateway system for realizing computing power assembly.
Background
With the continuous progress of the society, the rapid development and coverage of the internet of things, for example, under the application scenes of the internet of things such as smart cities, smart homes, smart buildings, smart security, smart traffic and the like, the increasingly abundant collection types and collection means of information and data and the popular application of artificial intelligent algorithms such as machine learning, face recognition and the like lead to the explosive increase of the data amount and the calculation amount which need to be processed, and the cloud-based internet of things solution can not meet the increasing demands of people.
In order to adapt to the development change that the data volume and the calculated volume which need to be processed are increased explosively, the edge computing architecture is more and more emphasized, and is used as the extension of the cloud and the transfer of the data processing authority to accelerate the data analysis speed and facilitate people to make decisions faster and better; the edge computing architecture is to place most of processing and computing at the edge of the internet of things close to the data source, namely a distributed processing and storage system structure, so that unnecessary data transmission and communication overhead in the processing process is reduced, and the real-time performance of response is improved.
Because the amount of data and computation involved in edge computation is highly variable rather than smooth and the distribution in the internet of things is unbalanced, the configuration requiring computation must be able to match the variation and distribution of the amount of data and computation.
In the edge computing mode, how to evaluate the computing power, how to configure the computing power, i.e., how to configure the processing capacity formed by the software and hardware resources for implementing data processing and computing, becomes a core problem.
Disclosure of Invention
Objects of the invention
Based on the above, to overcome at least one of the defects of the prior art, the present application provides an edge computing gateway system capable of performing computing power requirement evaluation and computing power configuration, and the present invention discloses the following technical solutions.
(II) technical scheme
The invention discloses an edge computing gateway system for realizing computing power assembly, which comprises: the system comprises a data source node, an edge computing gateway and a cloud collaborative computing server;
the data source nodes are provided with a plurality of nodes and used for sensing and generating data needing to be calculated and processed;
the edge computing gateway is in communication connection with a plurality of data source nodes and is used for acquiring, computing and processing data perceptively generated by the data source nodes and converting a network protocol;
the cloud-side collaborative computing server is in communication connection with the edge computing gateway, and is used for collaboratively computing the data generated by the data source node perception by the edge computing gateway and feeding back the processed result to the edge computing gateway.
In a possible implementation mode, a plurality of sensing devices are arranged on the data source node, and the sensing devices are sensing devices located at the front end of a scene under the application scenes of smart home, smart building, smart security, smart traffic, smart city management, smart government affairs, smart tourism, smart medical treatment and smart industrial automation.
In a possible implementation manner, a data processing module is disposed on the data source node, the data processing module includes a processing unit and a storage unit connected to the processing unit, and the processing unit is connected to the sensing device and configured to acquire different types of initial data acquired by the sensing device and store the initial data in the storage unit.
In a possible implementation manner, the data source node is further provided with a first data communication module connected to the data processing module, and the first data communication module is configured to send the initial data stored in the storage unit to the edge computing gateway.
In a possible implementation manner, the edge computing gateway includes a data storage module for storing the initial data sent by the first data communication module.
In one possible embodiment, the data storage module is divided into a resource management layer, and the resource management layer is used for resource management allocation of the data storage space of the edge computing gateway.
In a possible implementation manner, the data storage module is further divided into an intermediate layer, and the intermediate layer is used for storing streaming media data and data block data in different storage manners; wherein,
storing the storage mode for storing the streaming media data by adopting a stack structure;
and storing the data block data by adopting a data block pool structure.
In one possible implementation, the data storage module is further divided into an index layer for storing the structured data, the semi-structured data, and the structured index and the semi-structured index; wherein;
and storing the structured data, the semi-structured data, and the structured index and the semi-structured index by adopting an object unit structure.
In a possible implementation, the edge computing gateway further includes a computing load analysis module, configured to perform an evaluation of computing requirements on the data stored in the middle layer and the index layer; wherein,
for the structured data and the unstructured data, an evaluation of the computational power requirements may be performed with reference to their structured and unstructured indices;
an evaluation of the computational requirements may be performed on the unstructured data based on a saturation state of the stack structure storing the streaming media and a saturation state of the data chunk pool structure storing the data chunks.
In one possible embodiment, the edge computing gateway further comprises a computation power module for computing and processing the structured data, the semi-structured data, and unstructured data; wherein,
the computing force module adopts a computing processing framework of a CPU + GPU + an expansion processing unit; wherein,
the expansion processing unit is a certain number of XPUs, and the CPU controls the GPU and the certain number of XPUs according to the evaluation of the computational power load analysis module, and analyzes, processes and calculates the initial data.
(III) advantageous effects
The experimental equipment borrowing and returning management system and the experimental equipment borrowing and returning method have the following beneficial effects:
1. different types of data can be collected in real time through data source nodes deployed in different application scenes;
2. different load analyses are carried out on structured, semi-structured and unstructured data by arranging a calculation force load analysis module, so that the evaluation of calculation force requirements is realized;
3. the computing processing architecture of the CPU + the GPU + the expansion processing unit is adopted, and the CPU can better enable data to be processed reasonably and efficiently according to the computing power requirement obtained by the computing power load analysis module.
Drawings
The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining and illustrating the present invention and should not be construed as limiting the scope of the present invention.
FIG. 1 is an overall block diagram of an edge computing gateway system implementing computational force assembly according to the present disclosure;
FIG. 2 is a block diagram of an edge computing gateway architecture for an edge computing gateway system implementing computational power assembly in accordance with the present invention;
FIG. 3 is a block diagram of a data storage module of an edge computing gateway system implementing computational power assembly in accordance with the present disclosure;
FIG. 4 is a data source node structure diagram of an edge computing gateway system implementing computational power assembly according to the present invention;
FIG. 5 is a block diagram of a data processing module of an edge computing gateway system implementing computational power assembly according to the present invention.
Reference numerals: the method comprises the following steps of 1-a cloud collaborative computing server, 2-an edge computing gateway, 3-a data source node, 21-a second data communication module, 22-a data storage module, 23-a calculation load analysis module, 24-a calculation module, 221-an index layer, 222-a middle layer, 223-a resource management layer, 31-a first data communication module, 32-a data processing module, 321-a processing unit and 322-a storage unit.
Detailed Description
In order to make the implementation objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be described in more detail below with reference to the accompanying drawings in the embodiments of the present invention.
It should be noted that: in the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described are some embodiments of the present invention, not all embodiments, and features in embodiments and embodiments in the present application may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Herein, "first", "second", etc. are used only for distinguishing one from another, and do not indicate their importance degree, order, etc., for example, a first data communication module mentioned later may also be named as a second data communication module, and the second data communication module may also be named as a first data communication module.
The division of modules herein is merely a division of logical functions, and other divisions may be made in actual implementation, for example, a plurality of modules may be combined or integrated in another system. The modules, units and assemblies described as separate parts may be physically separated or not, so that part or all of them may be selected according to actual needs to implement the solutions of the embodiments.
An edge computing gateway system for implementing computing power configuration disclosed in the present invention is described in detail below with reference to fig. 1-5; as shown in fig. 1-5, an edge computing gateway system implementing computational force assembly includes: the system comprises a data source node 3, an edge computing gateway 2 and a cloud collaborative computing server 1; in the application scenes of the internet of things such as smart homes, smart buildings, smart security, smart traffic, smart city management, smart government affairs, smart tourism, smart medical treatment, smart industrial automation and the like (the application scenes of the internet of things listed herein do not represent all the application scenes of the internet of things, and can also be other existing application scenes of the internet of things), a plurality of data source nodes 3 are arranged according to the requirements of different application scenes of the internet of things and are used for sensing and generating data needing to be calculated and processed in various application scenes of the internet of things; the edge computing gateway 2 is in communication connection with a plurality of data source nodes 3 and is used for acquiring, computing and processing data generated by sensing of the data source nodes 3 and converting the data into a network protocol, wherein the edge computing gateway 2 has super-strong edge data computing capacity, integrates data acquisition, processing and execution, and can avoid the time delay defect and the problem of unsmooth remote communication caused by uploading and issuing of some data through the issuing of data processing permission of a cloud end, so that the processing capacity and the response speed of local Internet of things equipment are improved; the cloud-side collaborative computing server 1 is in communication connection with the edge computing gateway 2, the cloud-side collaborative computing server 1 is used for computing and processing data generated by sensing of the data source node 3 in cooperation with the edge computing gateway 2 and feeding back a processing result to the edge computing gateway 2, it needs to be explained that the data generated by sensing of the data source node 3 and required to be computed and processed are data of different types, and initial data collected by the data source node 3 is divided into the following data according to a data structure: structured data, semi-structured data, and unstructured data; dividing the data into stream media and data blocks according to the data form; wherein the structured data is represented as data in two-dimensional form, i.e. numbers and symbols, such as bank card numbers, telephone numbers, various addresses, article names, financial amounts, dates, etc.; the semi-structured data is generally in three formats of XML, HTML and JSON; the unstructured data has an irregular or incomplete data structure and no predefined data, and is not conveniently embodied by a database two-bit logic table, namely office documents, texts, pictures, various reports, images, audio or video information and the like in all formats.
In one embodiment, a plurality of sensing devices are disposed on the data source node 3, the sensing devices are sensing devices located at the front end of the site in the application scenarios of smart home, smart building, smart security, smart transportation, smart city management, smart government affairs, smart travel, smart medical treatment, and smart industrial automation, it should be noted that the sensing devices in the above application scenarios are the data acquisition device 1, the data acquisition device 2, the data. A temperature monitoring device, such as a thermostat for smart homes, a perception-type device to collect thermal energy data through a smart camera and temperature sensor; the water level monitoring device can be a water level monitoring device, for example, a sewer water level monitoring device applied to cities, and can be used as a sensing device for collecting urban sewer water level data; the traffic monitoring device can be a traffic monitoring device, such as a traffic monitoring device applied to intelligent traffic, and is used for acquiring vehicle data on a road section covered by the traffic monitoring device; may be a location monitoring device, such as a GPS applied to smart city management, for collecting location data of city management personnel within a grid; for example, the government affair self-service terminal is applied to intelligent government affairs and is used for collecting feedback information data of users; in addition, regarding the number of the data source nodes 3, the number is not limited in the embodiment of the present invention, and the required configuration may be performed according to the actual application requirement; in the above, the listed monitoring devices do not represent all sensing devices, and may also be other existing sensing devices with a data acquisition function, in this embodiment, the sensing devices are not limited, and the specific sensing devices are set according to an actual application scenario of the internet of things.
In a possible embodiment, a data processing module 32 is disposed on the data source node 3, the data processing module 32 includes a processing unit 321 for performing encapsulation processing on the structured data, the semi-structured data, and the unstructured data generated by sensing the sensing-type device in each application scenario, and a storage unit 322 connected to the processing unit 321 for storing initial data generated by sensing the sensing-type device in each application scenario of the internet of things, where the initial data is obtained by acquiring different types of initial data collected by each sensing-type device and storing the initial data in the storage unit 322, where the initial data stored in the storage unit 322 refers to data that is encapsulated only by the processing unit 321, and the storage unit 322 is connected directly to the sensing-type device in each application scenario of the internet of things for storing the initial data The data that is not processed by any other processing, for example, the sensing device connected to the processing unit 321 is a monitoring device (camera), the image data or video data collected by the monitoring device is sent to the processing unit 321 in the data processing module 32 connected to the monitoring device, the video data or image data is encapsulated by the processing unit 321, and then the initial data is directly stored in the storage unit 322.
In a possible implementation manner, the data source node 3 is further provided with a first data communication module 31 connected to the data processing module 32, where the first data communication module 31 is configured to send initial data stored in the storage unit 322 in the data processing module 32 to the edge computing gateway 2 for data processing, it should be noted that a communication manner between the data source node 3 and the edge computing gateway 2 is to receive and send data through an Internet of things located in the same local area, where the communication technology may adopt, for example, NB-IOT (Narrow Band-Internet of things) wireless communication technology, and this has the following advantages: the water level monitoring device has the advantages that the coverage area is wide (the water level monitoring device can cover the underground and can monitor the water level of a sewer and transmit monitored data to the edge computing gateway 2 through the communication mode), the capacity is large, the NB-IOT can support 10 thousands of connections in the communication range of a sector area, low delay sensitivity, ultralow equipment cost and low equipment power consumption are supported, the service life of a battery is long, the standby time of an NB-IOT terminal module can last for 10 years, the module cost is low, the cost of each module does not exceed 5 dollars, in addition, in the embodiment, the NB-IOT internet of things wireless communication technology can meet the requirement of the embodiment for carrying out internet of things communication, other existing internet of things communication technologies can also be adopted, and therefore the communication technology is not limited in the embodiment.
In a possible implementation, the edge computing gateway 2 includes a data storage module 22, and the data storage module 22 is configured to store initial data generated by perceptual device perception in each application scenario of the internet of things, which is encapsulated by the processing unit 321 in the data processing module 32, and then sent to the second data communication module 21 through the first data communication module 31 connected to the data processing module 32 in the data source node 3.
In a possible embodiment, the data storage module 22 is divided into a resource management layer 223, the resource management layer 223 is used for resource management allocation of the data storage space of the edge computing gateway 2, it should be noted that the data storage space herein may use a solid state disk and a mechanical hard disk with capacities capable of accommodating different types of data collected by the sensing-type device as the data storage space, and the resource management layer 223 is provided for better allocation and utilization of the storage space of the data storage module 22, so as to improve the utilization rate of the storage space.
In a possible embodiment, the data storage module 22 is further divided into an intermediate layer 222 and an index layer 221, wherein the intermediate layer 222 is configured to store streaming media data and data block data in different storage manners; the storage mode for storing the streaming media data adopts a stack structure for storage, and it should be noted that the streaming media includes audio stream, text stream, image stream, video stream, etc., the stack is a specific storage area or a register, one end is fixed, and the other end is floating; that is, all data is stored or fetched, and must be accessed according to the "first-in last-out" principle, where the size of the stack space is controlled and allocated by the resource management layer 223 in the data storage module 22, for example, the size of the stack space is 8 bytes (the 8 bytes are used for illustration only and are not used for limiting the size of the stack space, and the specific size of the stack space is set according to the actual requirement), then the resource management layer 223 allocates the corresponding size of the stack space according to the size of the stack space defined by the defined program code; in addition, the storage mode for storing the data block data adopts a structure of a data block pool, and the space size of the data block pool is controlled and allocated by the resource management layer 223.
In one possible implementation, the data storage module 22 is further divided into an index layer 221, and the index layer 221 is used for storing the structured data, the semi-structured data, and the structured index and the semi-structured index; it should be noted that the index structures of the Structured indexes and the semi-Structured indexes are designed by tree structures, for example, a B tree (Balanced-tree), a B tree variant B + tree, and a B + tree variant B + tree can be designed as tree structures in this embodiment, and hash structures are not used, for example, the average time complexity of HashMap Query/insertion/modification/deletion is o (1), wherein the hash is used for a single-line access Query, the hash is fastest, because only one record is queried each time, and the SQL (Structured Query Language database) requirements for sorting queries relate to grouping, querying, comparison, and the like, and the time complexity of the hash structure is reduced to o (n), because of the ordered nature of the tree, o (log) can still be maintained2n) high efficiency, and in addition, tree structure is selected as the index structure design to assist the later mentioned computational load analysis module 23 to calculate the gateway 2 data to the edgeThe calculation power required for processing is used for performing the calculation power requirement evaluation, and the details will be described in detail later.
In a possible implementation, the edge computing gateway 2 further includes a computing load analysis module 23, configured to perform an evaluation of computing requirements on the data stored in the intermediate layer 222 and the index layer 221; the evaluation of computational power requirements can be performed on the structured data and the unstructured data by referring to the structured indexes and the unstructured indexes of the structured data and the unstructured data, and it should be noted that the computational power refers to processing bearing capacity formed by software and hardware resources for realizing data processing and computation; the computation load analysis module 23 in the edge computation gateway 2 performs the evaluation of computation demand through the structured index and the semi-structured index, which may be embodied as, for example, performing binary search on a keyword ordered sequence in a node from a root node in the search of a B-tree (Balanced-tree), indicating the end if a search target is hit, otherwise, entering a child node in a range to which a query keyword belongs; repeating until the pointer of the corresponding child node is null or the child node is a leaf node, the computation load analysis module 23 performs evaluation on the computation requirement by the number of times of searching the key data of the data to be computed, the tree structure built by the index (i.e. the number of linked lists in the index storage) and the complexity.
In a possible embodiment, for the unstructured data, the evaluation of the computation power requirement may be performed according to a saturation state of a stack structure storing the streaming media and a saturation state of a data block pool structure storing the data blocks, where it should be noted that, the saturation state refers to that, the computation power load analysis module 23 analyzes the occupancy rate of the stack space and the occupancy rate of the data block pool to obtain the data amount to be processed, so as to achieve the evaluation of the computation power requirement, specifically, for the unstructured data such as streaming media and data blocks collected by the data source node 3 in various application scenarios of the internet of things, the delay of the edge computing gateway 2 for processing the streaming media and data blocks may be relatively large, and there is a problem that the unstructured data cannot be processed in real time, therefore, first, the initial data to be processed uploaded by the data source node 3 may be stored in the stack or the data block pool, then, the edge computing gateway 2 reads the initial data to be processed little by little in the subsequent data processing process, and then processes the data; at this time, the maximum capacity of the space size of the stack and the space size of the data block pool, that is, the maximum storage amount of the maximum unstructured initial data, the proportion occupied by the space of the unstructured initial data stored in the stack or the space of the data block pool is called a saturation state, for example, the size of the storage space of the stack is 8 bytes (the 8 bytes are used only for illustration and are not used for limiting the size of the storage space of the stack, and the specific size of the storage space is set according to the needs of the actual situation), 4 bytes of the storage space of the stack are occupied by the unstructured initial data to be processed uploaded by the data source node 3, then 50% of the storage space of the stack is occupied at this time, or 6 bytes of the storage space occupied by the stack are needed by the unstructured initial data to be processed uploaded by the data source node 3, and then 75% of the storage space of the stack is occupied at this, the computation load analysis module 23 evaluates the computation demand according to the occupancy rate of the stack space at this time, and if 75% of the occupancy rate is found, the computation demand is large because the amount of data to be processed is large at this time.
The calculation power requirement judgment of the calculation power load analysis module 23 on the data block pool is the same as the judgment on the stack calculation power requirement, and a detailed explanation is not given here, and the specific judgment method refers to the judgment on the stack calculation power requirement.
In one possible embodiment, the edge computing gateway 2 further includes a computation power module 24, the computation power module 24 being configured to compute and process structured data, semi-structured data, and unstructured data; the computing module 24 adopts a computing processing architecture of a CPU + GPU + an extended processing unit; wherein, the expansion Processing Unit is a certain amount of XPU, CPU (central Processing Unit) is used as the central node of the computing power module 24 to complete the task arrangement and computing power scheduling functions, and according to the evaluation of the computing power requirement made by the computing power load analysis module 23, the GPU (graphic Processing Unit) and a certain amount of XPU are controlled to analyze, process and calculate the initial data, wherein, the GPU carries out special audio and video data Processing in the service scene (such as intelligent security and intelligent transportation) facing the large amount of audio and video data Processing collected by the data source node 3, and can provide the functions of streaming media slice transcoding, computer vision image calculation acceleration, and the like, in addition, the expansion Processing Unit XPU part relates to the edge computing gateway 2 with a large amount of computing power placed at the front end to complete online incremental learning, computational power scheduling, and the like, Promote the task of derivation ability two major directions, in this angle of promotion derivation ability, then there are a plurality of key influence factors such as computational density, calculation power consumption again, so at this moment, can evaluate according to the calculation power demand of making of calculation power load analysis module 23, regard more XPU as extensible elasticity calculation power part, through knowing different task types, network complexity, network depth, modeling complexity etc., insert XPU calculation power module 24 into marginal computing gateway 2, improve the calculation and the scene adaptability of gateway, XPU here can be a certain quantity of CPU or a certain quantity of GPU, can also be some processors that are used for handling the specific data that perception type equipment gathered specially, for example, VPU (Video processor, Video Processing Unit) handles dynamic Video, carry out real-time coding and decoding.
Finally, in the whole data processing process, firstly, various types of data are acquired in various internet of things application scenes by various sensing devices with data acquisition functions, which are installed on the data source node 3, wherein the data acquired by the sensing devices can be a series of data such as sound data, video data, image data, text data and the like; and stores the initial data collected by each sensing-type device in the data processing module 32.
Then, through the communication between the first data communication module 31 and the second communication module 21, the initial data collected by the sensing-type device is sent to the data storage module 22 in the edge computing gateway 2 for classified storage, and the allocation of the storage space of the edge computing gateway 2 is realized through the resource management layer 223, for example, the middle layer 222 in the data storage module 22 is specially used for storing the storage areas of streaming media data and data block data; the indexing layer 221 in the data storage module 22 is dedicated to the storage of structured and semi-structured data and structured and semi-structured indexes.
Immediately before the computing power module 24 in the edge computing gateway 2 starts processing the acquired initial data, an evaluation of the execution computing power requirement is made by the computing power load analysis module 23 in the edge computing gateway 2 according to the stack structure of the middle tier 222 storing the streaming media data and the data block data and the saturation state of the data block pool, and the index structure of the structured index and the semi-structured index in the index tier 221 storing the structured data and the semi-structured data.
Secondly, the central processing unit CPU in the edge computing gateway 2 performs computation scheduling during data processing according to the evaluation of the execution computation demand made by the computation load analysis module 23, thereby reducing the time required for computation with a better and faster processing speed, and achieving a truly low latency.
Finally, cloud-side coordination is realized through the cloud-side collaborative computing server 1 and the edge computing gateway 2, that is, some computing tasks needing to be processed by the cloud-side collaborative computing server 1 are uploaded to the cloud-side collaborative computing server 1 by the edge computing gateway 2, the computing tasks uploaded by the edge computing gateway 2 are computed by the cloud-side collaborative computing server 1, and then the computing results of the computing tasks are fed back to the edge computing gateway 2 by the cloud-side collaborative computing server 1, so that computing coordination of the edge computing gateway 2 and the cloud-side collaborative computing server 1 in various types of data processing is realized for different computing tasks.
The invention has the advantages that the calculation power can be evaluated through the calculation power load analysis module 23 before the required calculation data is processed, then the calculation power scheduling is realized by the CPU according to the calculation power demand evaluation of the calculation power load analysis module 23 through the calculation processing architecture of the CPU + GPU + the expansion processing unit, and the data calculation processing speed is higher and more efficient.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (3)
1. An edge computing gateway system implementing computational force assembly, comprising: the system comprises a data source node, an edge computing gateway and a cloud collaborative computing server;
the data source nodes are provided with a plurality of nodes and used for sensing and generating data needing to be calculated and processed;
the data source node is provided with a data processing module, the data processing module comprises a processing unit and a storage unit connected with the processing unit, and the processing unit is connected with the perception type equipment and is used for acquiring different types of initial data acquired by the perception type equipment and storing the initial data in the storage unit;
the data source node is also provided with a first data communication module connected with the data processing module, and the first data communication module is used for sending the initial data stored in the storage unit to the edge computing gateway; the edge computing gateway is in communication connection with a plurality of data source nodes and is used for acquiring, computing and processing data perceptively generated by the data source nodes and converting a network protocol;
the edge computing gateway comprises a data storage module used for storing the initial data sent by the first data communication module;
the data storage module is divided into a resource management layer, and the resource management layer is used for resource management allocation of the data storage space of the edge computing gateway;
the data storage module is further divided into an intermediate layer, and the intermediate layer is used for storing the streaming media data and the data block data in the initial data in different storage modes; wherein, the storage mode for storing the streaming media data adopts a stack structure for storage; storing the data block data by adopting a data block pool structure;
the data storage module is further divided into an index layer, and the index layer is used for storing the structured data and the semi-structured data in the initial data, and the structured index and the semi-structured index; wherein; storing the structured data, the semi-structured data, and the structured index and the semi-structured index in an object unit structure;
the edge computing gateway also comprises a computing load analysis module which is used for evaluating the computing demand of the data stored in the middle layer and the index layer, and the edge computing gateway carries out computing scheduling when processing the data according to the evaluation; wherein,
the computing load analysis module evaluates the computing power requirement by looking up the times of searching key word data of the data to be computed, and tree structures built by indexes and complexity;
for unstructured data, evaluating the computational power requirement according to the saturation state of the stack structure for storing the streaming media and the saturation state of the data block pool structure for storing the data blocks, and analyzing the occupancy rate of the stack space and the occupancy rate of the data block pool by the computational power load analysis module to obtain the data volume to be processed, so that the evaluation of the computational power requirement is realized;
the cloud-side collaborative computing server is in communication connection with the edge computing gateway and is used for collaboratively computing the data generated by the data source node perception by the edge computing gateway and feeding back the processed result to the edge computing gateway.
2. The edge computing gateway system for implementing computational power configuration as claimed in claim 1, wherein a plurality of sensing devices are installed on the data source node, and the sensing devices are smart homes, smart buildings, smart security, smart traffic, smart city management, smart government affairs, smart tourism, smart medical treatment, and smart industrial automation applications located at front end of site.
3. The edge computing gateway system implementing computing power assembly of claim 1, wherein the edge computing gateway further comprises a computing power module for computing and processing the structured data, the semi-structured data, and unstructured data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010219671.2A CN111586091B (en) | 2020-03-25 | 2020-03-25 | Edge computing gateway system for realizing computing power assembly |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010219671.2A CN111586091B (en) | 2020-03-25 | 2020-03-25 | Edge computing gateway system for realizing computing power assembly |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111586091A CN111586091A (en) | 2020-08-25 |
| CN111586091B true CN111586091B (en) | 2021-03-19 |
Family
ID=72126083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010219671.2A Active CN111586091B (en) | 2020-03-25 | 2020-03-25 | Edge computing gateway system for realizing computing power assembly |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111586091B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112068964A (en) * | 2020-09-15 | 2020-12-11 | 杭州雾联科技有限公司 | Slice type edge computing force management method |
| CN112599233A (en) * | 2020-12-24 | 2021-04-02 | 汉锦科技（北京）有限公司 | Multi-sensor logic control central integrated control system for home medical treatment |
| CN113079530B (en) * | 2021-03-31 | 2022-05-27 | 广东电网有限责任公司电力调度控制中心 | Cloud edge collaborative operation and maintenance support system based on 5G slice |
| CN113064731B (en) * | 2021-06-03 | 2021-11-02 | 明品云(北京)数据科技有限公司 | Cloud-edge-architecture-based big data processing terminal device, processing method and medium |
| CN114501037B (en) * | 2021-12-24 | 2024-03-01 | 泰华智慧产业集团股份有限公司 | Video analysis method and system based on Bian Yun cooperative computing in intelligent pole scene |
| CN114866574A (en) * | 2022-03-28 | 2022-08-05 | 上海科技大学 | Passive Internet of things integrated system based on cloud platform |
| CN117389742B (en) * | 2023-11-10 | 2024-05-31 | 深圳市天鹤科技有限公司 | Edge computing method, device and storage medium for machine vision |
| CN117560250B (en) * | 2023-11-22 | 2024-06-18 | 重庆市华驰交通科技有限公司 | Application method, device and storage medium of intelligent gateway |
| CN117539647B (en) * | 2024-01-09 | 2024-04-12 | 四川华鲲振宇智能科技有限责任公司 | Task scheduling planning method and system based on edge computing gateway node attribute |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740026A (en) * | 2019-01-11 | 2019-05-10 | 深圳市中电数通智慧安全科技股份有限公司 | Smart city edge calculations platform and its management method, server and storage medium |
| CN110022234A (en) * | 2019-04-16 | 2019-07-16 | 中国人民解放军国防科技大学 | Method for realizing unstructured data sharing mechanism facing edge calculation |
| CN110247977A (en) * | 2019-06-17 | 2019-09-17 | 中国联合网络通信集团有限公司 | A kind of method and system of the data fusion based on edge calculations |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8185558B1 (en) * | 2010-04-19 | 2012-05-22 | Facebook, Inc. | Automatically generating nodes and edges in an integrated social graph |
| US9002773B2 (en) * | 2010-09-24 | 2015-04-07 | International Business Machines Corporation | Decision-support application and system for problem solving using a question-answering system |
| CN104820670B (en) * | 2015-03-13 | 2018-11-06 | 华中电网有限公司 | A kind of acquisition of power information big data and storage method |
| CN108363713A (en) * | 2017-12-20 | 2018-08-03 | 武汉烽火众智数字技术有限责任公司 | Video image information resolver, system and method |
| CN109067583A (en) * | 2018-08-08 | 2018-12-21 | 深圳先进技术研究院 | A kind of resource prediction method and system based on edge calculations |
| CN109388492B (en) * | 2018-10-09 | 2021-06-18 | 浙江工业大学 | Mobile block chain optimization calculation force distribution method based on simulated annealing in multi-edge calculation server scene |
| CN109379420B (en) * | 2018-10-10 | 2021-03-26 | 上海方融科技有限责任公司 | Comprehensive energy service platform system based on distributed architecture |
| CN109462652B (en) * | 2018-11-21 | 2021-06-01 | 杭州电子科技大学 | Terminal gateway load distribution method based on Hash algorithm in intelligent home system |
| CN109657003A (en) * | 2018-11-26 | 2019-04-19 | 深圳市玛尔仕文化科技有限公司 | A method of hardware data is directly accessed big data platform |
| CN110225075A (en) * | 2019-03-25 | 2019-09-10 | 北京快电科技有限公司 | A kind of building energy internet wisdom operation cloud operating system |
| CN110377278B (en) * | 2019-06-03 | 2023-11-03 | 杭州黑胡桃人工智能研究院 | Visual programming tool system based on artificial intelligence and Internet of things |
| CN110474839B (en) * | 2019-07-08 | 2021-09-03 | 冯瑞军 | Edge computing gateway device |
-
2020
- 2020-03-25 CN CN202010219671.2A patent/CN111586091B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740026A (en) * | 2019-01-11 | 2019-05-10 | 深圳市中电数通智慧安全科技股份有限公司 | Smart city edge calculations platform and its management method, server and storage medium |
| CN110022234A (en) * | 2019-04-16 | 2019-07-16 | 中国人民解放军国防科技大学 | Method for realizing unstructured data sharing mechanism facing edge calculation |
| CN110247977A (en) * | 2019-06-17 | 2019-09-17 | 中国联合网络通信集团有限公司 | A kind of method and system of the data fusion based on edge calculations |
Non-Patent Citations (1)
| Title |
|---|
| AI+IoT赋能智慧社区的现状和技术趋势;鲍敏;《人工智能》;20190210(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111586091A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111586091B (en) | Edge computing gateway system for realizing computing power assembly | |
| CN109739849B (en) | Data-driven network sensitive information mining and early warning platform | |
| CN111885040A (en) | Distributed network situation perception method, system, server and node equipment | |
| CN104820708B (en) | A kind of big data clustering method and device based on cloud computing platform | |
| CN105303456A (en) | Method for processing monitoring data of electric power transmission equipment | |
| CN102880854B (en) | Distributed processing and Hash mapping-based outdoor massive object identification method and system | |
| CN104820714A (en) | Mass small tile file storage management method based on hadoop | |
| CN102521386A (en) | Method for grouping space metadata based on cluster storage | |
| CN111159180A (en) | Data processing method and system based on data resource directory construction | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| CN103207920A (en) | Parallel metadata acquisition system | |
| Ma et al. | Design and implementation of smart city big data processing platform based on distributed architecture | |
| WO2021027331A1 (en) | Graph data-based full relationship calculation method and apparatus, device, and storage medium | |
| Luo et al. | Big-data analytics: challenges, key technologies and prospects | |
| CN107679127A (en) | Point cloud information parallel extraction method and its system based on geographical position | |
| CN112905571B (en) | Train rail transit sensor data management method and device | |
| CN111159107B (en) | Data processing method and server cluster | |
| CN113721856A (en) | Digital community management data storage system | |
| CN116226139B (en) | Distributed storage and processing method and system suitable for large-scale ocean data | |
| CN105912621A (en) | Area building energy consumption platform data storing and query method | |
| CN111190952B (en) | Method for extracting and persistence of multi-scale features of city portrait based on image pyramid | |
| CN111104416A (en) | Distributed electric power data management system | |
| CN112256730A (en) | Information retrieval method and device, electronic equipment and readable storage medium | |
| Shuai et al. | Memtv: a research on multi-level edge computing model for traffic video processing | |
| CN111813542A (en) | Load balancing method and device for parallel processing of large-scale graph analysis tasks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210208 Address after: 400010 no.50-6, 19 dapingzheng street, Yuzhong District, Chongqing Applicant after: Light control tesilian (Chongqing) Information Technology Co.,Ltd. Address before: 401329 no.2-1, building 6, No.39 Xinggu Road, Jiulongpo District, Chongqing Applicant before: CHONGQING TERMINUS TECHNOLOGIES Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |