CN108363778A - A kind of big data collection and analysis system and method based on information centre's network - Google Patents
A kind of big data collection and analysis system and method based on information centre's network Download PDFInfo
- Publication number
- CN108363778A CN108363778A CN201810136205.0A CN201810136205A CN108363778A CN 108363778 A CN108363778 A CN 108363778A CN 201810136205 A CN201810136205 A CN 201810136205A CN 108363778 A CN108363778 A CN 108363778A
- Authority
- CN
- China
- Prior art keywords
- nodes
- data
- mapper
- node
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The big data based on information centre's network that the present invention provides a kind of collecting and surveying system and method, including：Data collection step：The content name of required content given to different mapper nodes by master nodes, mapper nodes according to content name to information centre's net distribution interest packet, to obtain corresponding data；Content analysis step：The data that mapper nodes obtain are trained by reducer nodes to obtain final analysis model.The direct data acquisition of network layer and analysis may be implemented in the present invention.This greatly shortens data and generates to the process for obtaining analysis.Big data module can be delegated to network layer, and data are obtained and analyzed directly in data transmission procedure.More efficiently data can be analyzed, and the data being published on network are screened to a certain extent.
Description
Technical field
The present invention relates to technical field of data processing, and in particular, to a kind of big data receipts based on information centre's network
Set analysis system and method.
Background technology
With the hair at full speed of information technology, we can obtain information whenever and wherever possible by smart machine.It is diversified
The development of sensor technology and continually introduce the data resource that the intellectual technology of life brings magnanimity for us.It is more and more
People enjoy the value extracted from historical data and real time data, such as weather forecast, equity investment, intelligent medical, society
Turn over a finished item dynamic etc..But how to find required data in time from such large-scale data and analyze it becomes one
A main problem.The great development of internet and the new application constantly introduced, cause new demand from framework.In information
Heart network (ICN) is one of hope framework of Future network architectures.The internet of today focuses more on content retrieval rather than point
Point to-point communication.User no longer lies in the source of data, but is more desirable to obtain the data of needs as soon as possible.ICN frames exactly solve
It has determined this problem.Compared with traditional network, the network based on content is more suitable for magnanimity information pattern.
Present big data processing is mainly carried out in application layer.Mapreduce is as a kind of universal Distributed Calculation
Framework can accelerate the processing speed to large-scale data.HDFS in Hadoop then provides storage for mass data.Using
Layer deployment Hadoop can be to carrying out corresponding processing analysis by the data of the acquisitions such as web crawlers.But these data are only
The data being published on internet, and data just perceiving or having not been transmitted application layer be can not be by Hadoop
It obtains.
The related data for being directed to the big data analysis in information centre's network at present is less.There is research by information centre's net
Caching in network is dissolved into software defined network to realize big data analysis, but in the forwarding of data Layer mainly with software
Based on the pass-through mode for defining network (SDN), the caching that ICN is provided is merely to reduce the response time of content, not
Really realize the big data in information centre's network.
Invention content
For the defects in the prior art, the object of the present invention is to provide a kind of, and the big data based on information centre's network is received
Set analysis system and method.
According to a kind of big data collection and analysis method based on information centre's network provided by the invention, including：
Data collection step：The content name of required content is given to different mapper nodes by master nodes,
Mapper nodes according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis step：The data that mapper nodes obtain are trained to obtain by reducer nodes final
Analysis model.
Preferably, the data collection step specifically includes：
Step 101：By master nodes according to the request for connecing user node, corresponding is established in task tracker
Business；
Step 102：Idle mapper nodes are assigned the task to by task tracker；
Step 103：Interest packet is sent to adjacent ICN back end, ICN according to the task of distribution by mapper nodes
Back end is by forwarding interest packet to find the data needed for mapper nodes in a network and being transmitted to mapper nodes；
Step 104：The data broadcasting of acquisition is given to intermediate data node, the publication of intermediate data node by mapper nodes
Task gives free time reducer nodes；
Step 105：Interest packet number of request is sent to intermediate storage node according to the task of publication by reducer nodes
According to interest packet is transmitted to corresponding mapper nodes, mapper by intermediate storage node according to the data of mapper node broadcasts
Node passes data back corresponding reducer nodes by intermediate storage node；
Step 106：It completes that result of calculation is transmitted to master nodes after accordingly calculating by reducer nodes,
Master nodes pass result of calculation back user node.
Preferably, the content analysis step carries out content analysis using convolutional neural networks.
Preferably, further including node selection step：The selection of mapper nodes is carried out using K-means algorithms.
According to a kind of big data collection and analysis system based on information centre's network provided by the invention, including：
Data collection module：The content name of required content is given to different mapper nodes by master nodes,
Mapper nodes according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis module：The data that mapper nodes obtain are trained to obtain by reducer nodes final
Analysis model.
Preferably, the data collection module specifically includes：
By master nodes according to the request for connecing user node, corresponding task is established in task tracker；
Idle mapper nodes are assigned the task to by task tracker；
Interest packet is sent to adjacent ICN back end, ICN back end according to the task of distribution by mapper nodes
By forwarding interest packet to find the data needed for mapper nodes in a network and being transmitted to mapper nodes；
The data broadcasting of acquisition is given to intermediate data node by mapper nodes, and intermediate data node release tasks are to sky
Not busy reducer nodes；
Interest packet request data is sent to intermediate storage node according to the task of publication by reducer nodes, centre is deposited
Interest packet is transmitted to corresponding mapper nodes by storage node according to the data of mapper node broadcasts, and mapper nodes are by data
Corresponding reducer nodes are passed back by intermediate storage node；
It completes that result of calculation is transmitted to master nodes after accordingly calculating by reducer nodes, master nodes will
Result of calculation passes user node back.
Preferably, the content analysis module carries out content analysis using convolutional neural networks.
Preferably, further including node selection module：The selection of mapper nodes is carried out using K-means algorithms.
Compared with prior art, the present invention has following advantageous effect：
Network layer big data collects and surveys system and method and net may be implemented in information centre's network proposed by the invention
The direct data acquisition of network layers and analysis.This greatly shortens data and generates to the process for obtaining analysis.Big data module can
To be delegated to network layer, data are obtained and analyzed directly in data transmission procedure.It can more efficient logarithm
According to being analyzed, and the data being published on network are screened to a certain extent.
Description of the drawings
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the MapReduce configuration diagrams in information centre's network of the present invention；
Fig. 2 is the logical architecture schematic diagram of the present invention；
Fig. 3 is the overall network configuration diagram of the present invention；
Fig. 4 is the back end schematic diagram in the embodiment of the present invention；
Fig. 5 is node clustering and selected mapper nodes schematic diagram in the embodiment of the present invention；
Fig. 6 is effect diagram of the different node numbers of the present invention under different sub-clusterings.
Specific implementation mode
With reference to specific embodiment, the present invention is described in detail.Following embodiment will be helpful to the technology of this field
Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill of this field
For personnel, without departing from the inventive concept of the premise, several changes and improvements can also be made.These belong to the present invention
Protection domain.
A kind of big data based on information centre's network provided by the invention collects and surveys method, first proposed in information
Big data collects the logical architecture with analysis in heart network, and according to the routing mode of information centre's network and MapReduce
Computing architecture proposes specific data collection architecture.Then, the present invention proposes training convolutional neural networks in the architecture
(CNN) and the method that carries out further content analysis.Finally, for more efficient acquisition data, present invention employs K-means
Algorithm realizes the selection of mapper nodes.
The present invention is as follows：
Network layer big data in design information central site network is collected and analysis framework, and big data analysis framework is introduced and is believed
Central site network (ICN) is ceased, as shown in Figure 1.Information centre's network contains a large amount of data, including is stored in back end
Content-data and network status data (such as Buffer Utilization, data stream degree, time delay).The framework of the present invention can be to information
The offer that collects and analyzes of mass data greatly helps in central site network.By introducing MapReduce frameworks, tune can be passed through
Processing is collected to the data in network with different algorithms, specific framework is as shown in Figure 2.
MapReduce first can carry out categorised collection according to numerical nomenclature information to the data in network.For content
Data, data-gathering process can first carry out rude classification collection, after classification is further processed to data name.Finally,
It will obtain the name set of data needed for one group.And data handling procedure (by taking CNN as an example), it can directly be obtained by data name
Access is according to CNN is trained, to obtain final analysis model.For network status data, data-gathering process is being received
It can be directly for statistical analysis to its after collection.And according to analysis result, new network strategy can be made and be deployed to again
In network.
In data acquisition, interest packet is distributed to all the elements node of whole network by mapper nodes, to obtain
Take the content information of each nodal cache.The requirement provided according to user, specific name of the data acquisition in a large amount of contents
In filter out the name of user requested data.For network status data, MapReduce frameworks are after obtaining required data
Corresponding statistics is directly carried out to calculate.According to analysis result, we will be seen that whole network state, and corresponding to formulate to this
Strategy controls network.
Content analyzing process mainly makes a concrete analysis of the content being collected into, and there are many available for analysis method.
We choose CNN and carry out specific content analysis.The combination of CNN and tradition MapReduce frameworks, may be implemented parallel training,
Substantially reduce the training time.This advantage can also be embodied in the MapReduce frameworks in information centre's network.
In data-gathering process, it is known that the specific name of required content, so being had by distributing interest packet
Volume data content.By the data analysis of MapReduce processes, final calculated value is returned to use by big data analysis module
Family.
Content name is given different mapper nodes by Master nodes, and mapper nodes are distributed according to name to ICN emerging
Interest packet is to obtain specific data.Mapper nodes are trained CNN, what reducer nodes then obtained mapper nodes
As a result it is integrated to obtain final training pattern and carries out measuring and calculation, result of calculation is finally returned into user.
Specific steps are summarised as：
Data collection step：The content name of required content is given to different mapper nodes by master nodes,
Mapper nodes according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis step：The data that mapper nodes obtain are trained to obtain by reducer nodes final
Analysis model, content analysis step carry out content analysis using convolutional neural networks.
Data collection step specifically includes：
Step 101：By master nodes according to the request for connecing user node, corresponding is established in task tracker
Business；
Step 102：Idle mapper nodes are assigned the task to by task tracker；
Step 103：Interest packet is sent to adjacent ICN back end, ICN according to the task of distribution by mapper nodes
Back end is by forwarding interest packet to find the data needed for mapper nodes in a network and being transmitted to mapper nodes；
Step 104：The data broadcasting of acquisition is given to intermediate data node, the publication of intermediate data node by mapper nodes
Task gives free time reducer nodes；
Step 105：Interest packet number of request is sent to intermediate storage node according to the task of publication by reducer nodes
According to interest packet is transmitted to corresponding mapper nodes, mapper by intermediate storage node according to the data of mapper node broadcasts
Node passes data back corresponding reducer nodes by intermediate storage node；
Step 106：It completes that result of calculation is transmitted to master nodes after accordingly calculating by reducer nodes,
Master nodes pass result of calculation back user node.
What data-gathering process obtained is the name set for needing analyzed data.Name set is divided into more points with key
Value pair<Label, name>Form is distributed to different mapper.Mapper sends interest according to data name into CCN networks
Packet, and wait for back end that data packet is transmitted to mapper nodes.Mapper first pre-processes data, then carries out
CNN is trained.
CNN carries out feature extraction by convolution to image.Multiple convolutional layers extract different features respectively in the picture.
Each convolutional layer has the weight of oneself and biasing to carry out further operation to the result of previous convolutional layer.After convolutional layer
For pond layer.Pond layer compresses the characteristic pattern of input, on the one hand characteristic pattern is made to become smaller, and simplifies network calculations complexity；
On the one hand Feature Compression is carried out, main feature is extracted.There are mainly two types of pondization operations：Average pondization and maximum pond.Pass through to
After propagate, constantly coefficient is adjusted.After mapper obtains intermediate result, reducer collects intermediate result, by weight
Knots modification is combined, to obtain model to the end and be tested.
Node selection step：The selection of mapper nodes is carried out using K-means algorithms.
According to the communication process in network, the time used in transmission data between each node can be obtained.When passing through transmission
Between, to determine the cluster situation of network node.
With the back end in the node on behalf ICN in two-dimensional coordinate.Euclidean distance between node represents transmission time.It is first
Initial Centroid is first selected, the distance by calculating each node to Centroid determines initial cluster situation.By not
Disconnected iteration finds final Centroid and determines node clustering situation.
We emulate the Algorithms of Selecting of mapper nodes.
First, 900 points are generated at random in two-dimensional coordinate system as CCN back end, as shown in Figure 4.We will
These nodes are divided into 5 clusters.In each cluster, the back end for having selected distance center point nearest is as mapper nodes, such as
Shown in Fig. 5.In Name-Getting Process, each mapper collects the data letter stored on each cluster interior joint respectively
Breath, and these data informations are analyzed, the specifying information for needing data is filtered out, is carried out again to obtain specific data
Further analyzing processing.
Fig. 6 is demonstrated by the case of 300 nodes, 900 nodes and 1500 nodes, corresponding to different sub-clustering numbers
Back end to each mapper nodes average distance.Due to the mapping that point-to-point transmission Euclidean distance is transmission time, Wo Menfa
Now the quantity of back end is little on being influenced to the transmission time of mapper in a certain range, and the mapper nodes the more more have
Conducive to the reduction of transmission time.
It is straight that network layer may be implemented in network layer big data collection and analysis framework in information centre's network proposed by the invention
The data acquisition connect and analysis.This greatly shortens data and generates to the process for obtaining analysis.Big data module can be by under
It is put into network layer, data are obtained and analyzed directly in data transmission procedure.More efficiently data can be carried out
Analysis, and the data being published on network are screened to a certain extent.
On the basis of a kind of above-mentioned big data collection and analysis method based on information centre's network, the present invention also provides one
Big data of the kind based on information centre's network collects and surveys system, including：
Data collection module：The content name of required content is given to different mapper nodes by master nodes,
Mapper nodes according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis module：The data that mapper nodes obtain are trained to obtain by reducer nodes final
Analysis model.The content analysis module carries out content analysis using convolutional neural networks.
The data collection module specifically includes：
By master nodes according to the request for connecing user node, corresponding task is established in task tracker；
Idle mapper nodes are assigned the task to by task tracker；
Interest packet is sent to adjacent ICN back end, ICN back end according to the task of distribution by mapper nodes
By forwarding interest packet to find the data needed for mapper nodes in a network and being transmitted to mapper nodes；
The data broadcasting of acquisition is given to intermediate data node by mapper nodes, and intermediate data node release tasks are to sky
Not busy reducer nodes；
Interest packet request data is sent to intermediate storage node according to the task of publication by reducer nodes, centre is deposited
Interest packet is transmitted to corresponding mapper nodes by storage node according to the data of mapper node broadcasts, and mapper nodes are by data
Corresponding reducer nodes are passed back by intermediate storage node；
It completes that result of calculation is transmitted to master nodes after accordingly calculating by reducer nodes, master nodes will
Result of calculation passes user node back.
Node selection module：The selection of mapper nodes is carried out using K-means algorithms.
One skilled in the art will appreciate that in addition to realizing system provided by the invention in a manner of pure computer readable program code
It, completely can be by the way that method and step be carried out programming in logic come so that the present invention provides and its other than each device, module, unit
System and its each device, module, unit with logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedding
Enter the form of the controller that declines etc. to realize identical function.So system provided by the invention and its every device, module, list
Member is considered a kind of hardware component, and also may be used for realizing the device of various functions, module, unit to include in it
To be considered as the structure in hardware component；It can also will be considered as realizing the device of various functions, module, unit either real
The software module of existing method can be the structure in hardware component again.
Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make a variety of changes or change within the scope of the claims, this not shadow
Ring the substantive content of the present invention.In the absence of conflict, the feature in embodiments herein and embodiment can arbitrary phase
Mutually combination.
Claims (8)
1. a kind of big data based on information centre's network collects and surveys method, which is characterized in that including：
Data collection step：The content name of required content is given to different mapper nodes, mapper by master nodes
Node according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis step：The data that mapper nodes obtain are trained by reducer nodes to obtain final analysis
Model.
2. the big data according to claim 1 based on information centre's network collects and surveys method, which is characterized in that described
Data collection step specifically includes：
Step 101：By master nodes according to the request for connecing user node, corresponding task is established in task tracker；
Step 102：Idle mapper nodes are assigned the task to by task tracker；
Step 103：Interest packet is sent to adjacent ICN back end, ICN data according to the task of distribution by mapper nodes
Node is by forwarding interest packet to find the data needed for mapper nodes in a network and being transmitted to mapper nodes；
Step 104：The data broadcasting of acquisition is given to intermediate data node, intermediate data node release tasks by mapper nodes
To idle reducer nodes；
Step 105：Interest packet request data is sent to intermediate storage node according to the task of publication by reducer nodes, in
Between memory node interest packet is transmitted to corresponding mapper nodes according to the data of mapper node broadcasts, mapper nodes will
Data pass corresponding reducer nodes back by intermediate storage node；
Step 106：It completes that result of calculation is transmitted to master nodes, master sections after accordingly calculating by reducer nodes
Point passes result of calculation back user node.
3. the big data according to claim 2 based on information centre's network collects and surveys method, which is characterized in that described
Content analysis step carries out content analysis using convolutional neural networks.
4. the big data according to claim 2 based on information centre's network collects and surveys method, which is characterized in that also wrap
Include node selection step：The selection of mapper nodes is carried out using K-means algorithms.
5. a kind of big data based on information centre's network collects and surveys system, which is characterized in that including：
Data collection module：The content name of required content is given to different mapper nodes, mapper by master nodes
Node according to content name to information centre's net distribution interest packet, to obtain corresponding data；
Content analysis module：The data that mapper nodes obtain are trained by reducer nodes to obtain final analysis
Model.
6. the big data according to claim 5 based on information centre's network collects and surveys system, which is characterized in that described
Data collection module specifically includes：
By master nodes according to the request for connecing user node, corresponding task is established in task tracker；
Idle mapper nodes are assigned the task to by task tracker；
Interest packet is sent to adjacent ICN back end according to the task of distribution by mapper nodes, ICN back end passes through
Forwarding interest packet finds the data needed for mapper nodes and is transmitted to mapper nodes in a network；
The data broadcasting of acquisition is given to intermediate data node by mapper nodes, and intermediate data node release tasks are to the free time
Reducer nodes；
Interest packet request data, intermediate storage section are sent to intermediate storage node according to the task of publication by reducer nodes
Interest packet is transmitted to corresponding mapper nodes by point according to the data of mapper node broadcasts, and mapper nodes pass through data
Intermediate storage node passes corresponding reducer nodes back；
It completes that result of calculation is transmitted to master nodes after accordingly calculating by reducer nodes, master nodes will calculate
As a result user node is passed back.
7. the big data according to claim 6 based on information centre's network collects and surveys system, which is characterized in that described
Content analysis module carries out content analysis using convolutional neural networks.
8. the big data according to claim 6 based on information centre's network collects and surveys system, which is characterized in that also wrap
Include node selection module：The selection of mapper nodes is carried out using K-means algorithms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810136205.0A CN108363778B (en) | 2018-02-09 | 2018-02-09 | Big data collecting and analyzing system and method based on information center network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810136205.0A CN108363778B (en) | 2018-02-09 | 2018-02-09 | Big data collecting and analyzing system and method based on information center network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108363778A true CN108363778A (en) | 2018-08-03 |
| CN108363778B CN108363778B (en) | 2020-11-24 |
Family
ID=63005289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810136205.0A Active CN108363778B (en) | 2018-02-09 | 2018-02-09 | Big data collecting and analyzing system and method based on information center network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108363778B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080027941A1 (en) * | 2006-07-28 | 2008-01-31 | International Business Machines Corporation | Method and System For Providing A Searchable Virtual Information Center |
| CN103886508A (en) * | 2014-03-03 | 2014-06-25 | 中标软件有限公司 | Mass farmland data monitoring method and system |
| CN104754687A (en) * | 2013-12-31 | 2015-07-01 | 华为技术有限公司 | Information sending method and device |
| CN104919758A (en) * | 2013-01-07 | 2015-09-16 | 华为技术有限公司 | Information centric networking based service centric networking |
| CN105426407A (en) * | 2015-11-02 | 2016-03-23 | 浪潮软件集团有限公司 | Web data acquisition method based on content analysis |
| CN106202399A (en) * | 2016-07-11 | 2016-12-07 | 浪潮软件集团有限公司 | Method for implementing data management system of big data |
| CN106789324A (en) * | 2017-01-09 | 2017-05-31 | 上海轻维软件有限公司 | FTP distributed acquisition methods based on MapReduce |
-
2018
- 2018-02-09 CN CN201810136205.0A patent/CN108363778B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080027941A1 (en) * | 2006-07-28 | 2008-01-31 | International Business Machines Corporation | Method and System For Providing A Searchable Virtual Information Center |
| CN104919758A (en) * | 2013-01-07 | 2015-09-16 | 华为技术有限公司 | Information centric networking based service centric networking |
| CN104754687A (en) * | 2013-12-31 | 2015-07-01 | 华为技术有限公司 | Information sending method and device |
| CN103886508A (en) * | 2014-03-03 | 2014-06-25 | 中标软件有限公司 | Mass farmland data monitoring method and system |
| CN105426407A (en) * | 2015-11-02 | 2016-03-23 | 浪潮软件集团有限公司 | Web data acquisition method based on content analysis |
| CN106202399A (en) * | 2016-07-11 | 2016-12-07 | 浪潮软件集团有限公司 | Method for implementing data management system of big data |
| CN106789324A (en) * | 2017-01-09 | 2017-05-31 | 上海轻维软件有限公司 | FTP distributed acquisition methods based on MapReduce |
Non-Patent Citations (2)
| Title |
|---|
| JALIYA EKANAYAKE 等: "MapReduce for Data Intensive Scientific Analyses", 《2008 IEEE FOURTH INTERNATIONAL CONFERENCE ON ESCIENCE》 * |
| 梁芳芳: "内容中心网络用户兴趣数据分析与处理系统研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108363778B (en) | 2020-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN207731321U (en) | Hardware accelerator engine | |
| CN103310012B (en) | A kind of distributed network crawler system | |
| Hossain et al. | Improving consumer satisfaction in smart cities using edge computing and caching: A case study of date fruits classification | |
| CN103345514B (en) | Streaming data processing method under big data environment | |
| CN106790718A (en) | Service call link analysis method and system | |
| CN107391719A (en) | Distributed stream data processing method and system in a kind of cloud environment | |
| CN108269224A (en) | Reconfigurable interconnection | |
| CN103886508B (en) | The monitoring method and system of magnanimity farm field data | |
| CN103077070B (en) | Cloud computing management system and management method for cloud computing systems | |
| CN107317724A (en) | Data collecting system and method based on cloud computing technology | |
| CN111614769B (en) | Behavior intelligent analysis engine system of deep learning technology and control method | |
| WO2023093724A1 (en) | Neural network model processing method and device | |
| CN101123613A (en) | A coding method and device and system for multi-dimension address | |
| CN107046557A (en) | The intelligent medical calling inquiry system that dynamic Skyline is inquired about under mobile cloud computing environment | |
| Vu et al. | Distributed adaptive model rules for mining big data streams | |
| CN110264407A (en) | Image Super-resolution model training and method for reconstructing, device, equipment and storage medium | |
| CN108320168A (en) | A kind of data analysing method and device | |
| CN114886404B (en) | Electronic equipment, device and storage medium | |
| CN107453948A (en) | The storage method and system of a kind of network measurement data | |
| CN109416688A (en) | Method and system for flexible high performance structured data processing | |
| CN106909624A (en) | A kind of real-time sorting consistence method of mass data | |
| CN108363778A (en) | A kind of big data collection and analysis system and method based on information centre's network | |
| Endler et al. | Stream-Based Reasoning for IoT Applications—Proposal of Architecture and Analysis of Challenges | |
| Song et al. | Towards modeling large-scale data flows in a multidatacenter computing system with petri net | |
| Wang et al. | C3Meta: A Context-Aware Cloud-Edge-End Collaboration Framework Toward Green Metaverse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |