CN108282378A - A kind of method and apparatus of monitoring traffic in network - Google Patents
A kind of method and apparatus of monitoring traffic in network Download PDFInfo
- Publication number
- CN108282378A CN108282378A CN201710007945.XA CN201710007945A CN108282378A CN 108282378 A CN108282378 A CN 108282378A CN 201710007945 A CN201710007945 A CN 201710007945A CN 108282378 A CN108282378 A CN 108282378A
- Authority
- CN
- China
- Prior art keywords
- task
- group system
- network flow
- daily record
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/06—Generation of reports
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the present application discloses a kind of method of monitoring traffic in network.The method includes：Monitoring system obtains the task scheduling daily record that task submits equipment to be generated for the task when task is submitted；Monitoring system is based on the task scheduling daily record, counts the network flow across cluster in the distributed file system；Monitoring system exports the statistical result of the network flow.Wherein, the task that the task submits equipment to be used to handle in distributed file system is submitted, and the task scheduling daily record includes that the task indicates the first data volume that the first group system is read from the second group system.Pass through method provided by the embodiments of the present application, the case where largely across company-data read operation occupies a large amount of network bandwidths in distributed file system can be prompted in advance when distributed file system provides the services such as off-line calculation, to avoid network bandwidth from largely being occupied by across company-data read operation.In addition, the embodiment of the present application also discloses a kind of equipment of monitoring traffic in network.
Description
Technical field
This application involves network data processing technique, more particularly to a kind of method and apparatus of monitoring traffic in network.
Background technology
In order to adapt to the magnanimity job requirements of big data processing platform, distributed file system can provide multiple for counting
According to the group system of storage.That is, the data in distributed file system be assigned to multiple and different group systems into
Row storage.When providing the services such as off-line calculation, the data read operation in distributed file system needs across group system reality
It is existing.Specifically, it is assumed that group system A and group system B is two cluster systems for data storage in distributed file system
System, group system A can be from the data being stored in group system B be read in group system B, group system B can also be from collection
Group's system A reads the data for being stored in group system A.
Between each group system connected by the network equipment in distributed file system, therefore, across the number of cluster
Network bandwidth can be occupied according to read operation.In the prior art, distributed file system is when providing the services such as off-line calculation, sometimes
Meeting occupied a large amount of network bandwidths because of largely across company-data read operation, at this point, distributed file system is normal
Operation will be affected.
Invention content
The technical problem to be solved in the embodiments of the present application is that a kind of method and apparatus of monitoring traffic in network are provided, with
The case where largely across company-data read operation occupies a large amount of network bandwidths in distributed file system is enable to occur it
It is preceding to prompt in advance, to avoid network bandwidth from largely being occupied by across company-data read operation, avoid distributed file system
Normal operation is affected.
In a first aspect, the embodiment of the present application provides a kind of method of monitoring traffic in network, it is applied to monitoring system, packet
It includes：
Acquisition task submits the task scheduling daily record that equipment is generated when task is submitted for the task, the task to submit
The task that equipment is used to handle in distributed file system is submitted, and the task scheduling daily record includes first collection of task instruction
The first data volume that group's system is read from the second group system；
Based on the task scheduling daily record, count the network flow across cluster in the distributed file system, it is described across
The network flow of cluster includes the network flow between first group system and second group system；
Export the statistical result of the network flow.
Optionally, the task scheduling daily record further includes that the task indicates first group system from third cluster system
The third data volume that the second data volume and second group system read in system is read from the third group system, institute
State the network flow across cluster further include network flow between first group system and the third group system and
Network flow between second group system and the third group system.
Optionally, described to be based on the task scheduling daily record, count the network across cluster in the distributed file system
Flow, including：
First data volume is identified from the task scheduling daily record；
According to first data volume, the first cluster system described in the distributed file system in statistics specified time
Network flow between system and second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
Optionally, described to be based on the task scheduling daily record, count the network across cluster in the distributed file system
Flow, specially：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as
Overall network flow involved in the task execution process.
Optionally, further include：
The statistical result of the network flow is saved in data persistence system.
Optionally, further include：
Meet prerequisite in response to the statistical result, exports warning message.
Optionally, record has the identifying of the task, the mark of first group system in the task scheduling daily record
Know, the mark of second group system and first data volume, wherein the mark of second group system and described the
One data volume has correspondence.
Second aspect, the embodiment of the present application provide a kind of method of monitoring traffic in network, are applied to task and submit equipment,
The task that the task submits equipment to be used to handle in distributed file system is submitted；
The method includes：
In response to the submission instruction of task, task scheduling daily record is generated for the task；
The task scheduling daily record is sent to monitoring system, so that the monitoring system is based on the task scheduling daily record,
The network flow across cluster in the distributed file system is counted, and exports the statistical result of the network flow；
Wherein, the task scheduling daily record includes that the task indicates that the first group system is read from the second group system
The first data volume, the network flow across cluster includes between first group system and second group system
Network flow.
Optionally, the task scheduling daily record further includes that the task indicates first group system from third cluster system
The third data volume that the second data volume and second group system read in system is read from the third group system, institute
State the network flow across cluster further include network flow between first group system and the third group system and
Network flow between second group system and the third group system.
Optionally, record has the identifying of the task, the mark of first group system in the task scheduling daily record
Know, the mark of second group system and first data volume, wherein the mark of second group system and described the
One data volume has correspondence.
The third aspect, the embodiment of the present application provide a kind of device of monitoring traffic in network, are configured at monitoring system, packet
It includes：
Acquiring unit, the task scheduling day for submitting equipment to be generated for the task when task is submitted for obtaining task
Will, the task that the task submits equipment to be used to handle in distributed file system are submitted, and the task scheduling daily record includes institute
It states task and indicates the first data volume that the first group system is read from the second group system；
Statistic unit counts the net across cluster in the distributed file system for being based on the task scheduling daily record
Network flow, the network flow across cluster include the network flow between first group system and second group system
Amount；
First output unit, the statistical result for exporting the network flow.
Optionally, the task scheduling daily record further includes that the task indicates first group system from third cluster system
The third data volume that the second data volume and second group system read in system is read from the third group system, institute
State the network flow across cluster further include network flow between first group system and the third group system and
Network flow between second group system and the third group system.
Optionally, the statistic unit, is specifically used for：
First data volume is identified from the task scheduling daily record；
According to first data volume, the first cluster system described in the distributed file system in statistics specified time
Network flow between system and second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
Optionally, the statistic unit, is specifically used for：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as
Overall network flow involved in the task execution process.
Optionally, further include：
Storage unit, for the statistical result of the network flow to be saved in data persistence system.
Optionally, further include：
Second output unit exports warning message for meeting prerequisite in response to the statistical result.
Optionally, record has the identifying of the task, the mark of first group system in the task scheduling daily record
Know, the mark of second group system and first data volume, wherein the mark of second group system and described the
One data volume has correspondence.
Fourth aspect, the embodiment of the present application provide a kind of device of monitoring traffic in network, are configured at task and submit equipment,
The task that the task submits equipment to be used to handle in distributed file system is submitted；
Described device includes：
Generation unit, instructs for the submission in response to task, and task scheduling daily record is generated for the task；
Transmission unit, for sending the task scheduling daily record to monitoring system, so that the monitoring system is based on described
Task scheduling daily record counts the network flow across cluster in the distributed file system, and exports the system of the network flow
Count result；
Wherein, the task scheduling daily record includes that the task indicates that the first group system is read from the second group system
The first data volume, the network flow across cluster includes between first group system and second group system
Network flow.
Optionally, the task scheduling daily record further includes that the task indicates first group system from third cluster system
The third data volume that the second data volume and second group system read in system is read from the third group system, institute
State the network flow across cluster further include network flow between first group system and the third group system and
Network flow between second group system and the third group system.
Optionally, record has the identifying of the task, the mark of first group system in the task scheduling daily record
Know, the mark of second group system and first data volume, wherein the mark of second group system and described the
One data volume has correspondence.
5th aspect, the embodiment of the present application provide a kind of system of monitoring traffic in network, including：Monitoring system, task
Submit equipment and distributed file system；The task that the task submits equipment to be used to handle in distributed file system is submitted；
The distributed file system includes at least the first group system and the second group system；
The task submits equipment, is instructed for the submission in response to task, and task scheduling daily record is generated for the task；
Wherein, the task scheduling daily record includes that the task indicates the first number that the first group system is read from the second group system
According to amount；
The monitoring system, task of submitting equipment to be generated for the task when task is submitted for obtaining the task
Plan daily record, be based on the task scheduling daily record, count the network flow across cluster in the distributed file system, exports institute
State the statistical result of network flow；Wherein, the network flow across cluster includes first group system and described second
Network flow between group system.
Optionally, the distributed file system further includes third group system；
The task scheduling daily record further includes that the task indicates that first group system is read from third group system
The third data volume that the second data volume and second group system taken is read from the third group system, it is described across collection
The network flow of group further includes network flow between first group system and the third group system and described the
Network flow between two group systems and the third group system.
Optionally, in order to count the network flow in the distributed file system across cluster, the monitoring system is specific
For：
First data volume is identified from the task scheduling daily record；
According to first data volume, the first cluster system described in the distributed file system in statistics specified time
Network flow between system and second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
Optionally, in order to count the network flow in the distributed file system across cluster, the monitoring system is specific
For：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as
Overall network flow involved in the task execution process.
Optionally, the monitoring system is additionally operable to：
The statistical result of the network flow is saved in data persistence system.
Optionally, the monitoring system is additionally operable to：
Meet prerequisite in response to the statistical result, exports warning message.
Optionally, record has the identifying of the task, the mark of first group system in the task scheduling daily record
Know, the mark of second group system and first data volume, wherein the mark of second group system and described the
One data volume has correspondence.
Compared with prior art, the application has the following advantages：
According to the technical solution of the embodiment of the present application, task submits equipment when task is submitted, in response to the task
It is that the task generates task scheduling daily record and is sent to monitoring system to submit instruction, wherein the task is related to the first cluster
System reads the operation of data from the second group system, and record has the task instruction first in the task scheduling daily record
The data volume that group system is read from the second group system.Monitoring system is according to the task scheduling daily record got, statistical
Across the network flow of cluster in cloth file system, and export the statistical result of the network flow.It can be seen that due to task
Plan daily record be generate and send to carry out network flow statistic to monitoring system when task is submitted, so, distributed literary
Before the practical execution task of part system, monitoring system can export the statistical result of the network flow based on the task, therefore,
Distributed file system can be predicted when providing the services such as off-line calculation and export across the company-data read operation that will be executed
Required network flow, to prompt in advance in distributed file system, largely across company-data read operation occupies a large amount of nets
The case where network bandwidth, is largely occupied to avoid network bandwidth by across company-data read operation, to avoid distributed field system
The normal operation of system is affected.
Description of the drawings
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments described in application, for those of ordinary skill in the art, without creative efforts,
Other drawings may also be obtained based on these drawings.
Fig. 1 is the network system block schematic illustration involved by an application scenarios in the embodiment of the present application；
Fig. 2 is a kind of flow diagram of the method for monitoring traffic in network in the embodiment of the present application；
Fig. 3 is a kind of flow diagram of the method for monitoring traffic in network in the embodiment of the present application；
Fig. 4 is a kind of structural schematic diagram of the device of monitoring traffic in network in the embodiment of the present application；
Fig. 5 is a kind of structural schematic diagram of the device of monitoring traffic in network in the embodiment of the present application；
Fig. 6 is a kind of structural schematic diagram of the system of monitoring traffic in network in the embodiment of the present application.
Specific implementation mode
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, technical solutions in the embodiments of the present application are clearly and completely described, it is clear that described embodiment is only this
Apply for a part of the embodiment, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art exist
The every other embodiment obtained under the premise of creative work is not made, shall fall in the protection scope of this application.
The application can be used in numerous general or special purpose computing system environments or configuration.Such as：Personal computer, service
Device computer, handheld device or portable device, laptop device, multicomputer system, microprocessor-based system, top set
Box, programmable consumer-elcetronics devices, network PC, minicomputer, mainframe computer including any of the above system or equipment
Distributed computing environment etc..
The application can describe in the general context of computer-executable instructions executed by a computer, such as program
Module.Usually, program module includes routines performing specific tasks or implementing specific abstract data types, program, object, group
Part, data structure etc..The application can also be put into practice in distribution counts environment, in these distributed computing environments, by leading to
Cross communication network and connected remote processing devices execute task.In a distributed computing environment, program module can position
In the local and remote computer storage media including storage device.
Present inventor passes through the study found that in the prior art, distributed file system is providing off-line calculation etc.
When service, it sometimes appear that largely across company-data read operation.Due to being connected by the network equipment between each group system,
Largely across company-data read operation can occupy a large amount of network bandwidth between each group system for these, therefore, distributed text
The normal operation of part system will influence.
To solve the above-mentioned problems, in the embodiment of the present application, it is contemplated that across company-data read operation is held in task
Operation during row, and when task is submitted, across company-data read operation has not been performed, across company-data read operation
Corresponding network flow does not occur in distributed file system yet, can be that task generation is appointed when task is submitted therefore
Business plan daily record, can record the corresponding network flow of across company-data operation of task instruction in task scheduling daily record,
Therefore, before task execution, the task scheduling daily record based on each task can count in distributed file system in advance
Network flow across cluster and output, therefore, distributed file system can be predicted and be exported when providing the services such as off-line calculation
The required network flow of across company-data read operation that will be executed, it is a large amount of in distributed file system to prompt in advance
Across company-data read operation occupies the case where a large amount of network bandwidths, big by across company-data read operation to avoid network bandwidth
Amount occupies, to avoid the normal operation of distributed file system from being affected.
For example, the embodiment of the present application for example can be applied in network system as shown in Figure 1.In the network system
With multiple group systems 103, data read operation, i.e. a group system can be carried out between different group systems 103
103 can read data from another group system 103.The network system is indicated with the first group system and the second group system
The different group system 103 of any two in system a, it is assumed that there are task relates to the first group system from the second cluster
The operation of data is read in system, task submits equipment 101 to appoint for task generation in response to the submission instruction of the task
Business plan daily record is simultaneously sent to monitoring system 102, wherein record has the task instruction first in the task scheduling daily record
The data volume that group system is read from the second group system.Monitoring system 102 is according to the task scheduling daily record got, statistics
Across the network flow of cluster in distributed file system, and export the statistical result of the network flow, wherein described across cluster
Network flow include network flow between first group system and second group system.
After describing the main thought of the application, below in conjunction with the accompanying drawings, the various unrestricted of the application is described in detail
Property embodiment.
Referring to Fig. 2, a kind of flow diagram of the method for monitoring traffic in network in the embodiment of the present application is shown.In this reality
It applies in example, the method can be applied to monitoring system, i.e., is executed by the monitoring system.The method for example can specifically wrap
Include following steps：
Step 201 obtains the task scheduling daily record that task submits equipment to be generated for the task when task is submitted, described
The task that task submits equipment to be used to handle in distributed file system is submitted, and the task scheduling daily record includes that the task refers to
Show the first data volume that the first group system is read from the second group system.
Distributed file system can respond and be related to across company-data read operation when providing the services such as off-line calculation
Task.In the submission request for receiving the task, task submits equipment that can be submitted to the task.Wherein, appoint
Business submits equipment when submitting the task, and it is that the task generates task scheduling that can be instructed in response to the submission of the task
Daily record is simultaneously sent to monitoring system.
In the present embodiment, can record that the task is related in the task scheduling daily record of the task one or
Multiple data volumes across company-data read operation.Assuming that needing the first group system from second during the task execution
The first data volume is read in group system, then the first group system pair can be recorded in the task scheduling daily record of the task
The data read operation of two group systems and its corresponding first data volume.Assuming that in addition to needing during the task execution
First group system is read from the second group system except the first data volume, it is also necessary to which the first group system is from third cluster system
The second data volume is read in system, then can also include that the task indicates described first in the task scheduling daily record of the task
The second data volume that group system is read from third group system can also be remembered in the task scheduling daily record of the task
Record data read operation and its corresponding second data volume of first group system to third group system.It assumes again that at described
In addition to needing the first group system to read the first data volume and the first cluster system from the second group system in business implementation procedure
System is read from third group system except the second data volume, it is also necessary to which the second group system reads the from third group system
Three data volumes, then can also include in the task scheduling daily record of the task task indicate second group system from
The third data volume read in third group system can also record the second cluster in the task scheduling daily record of the task
Data read operation and its corresponding third data volume of the system to third group system.Certainly, in order to more fully predict point
Network traffic conditions in cloth file system, all across the company-data read operations and its corresponding number that the task is related to
It can be recorded in the task scheduling daily record of the task according to amount.
It is understood that for a task, all across company-data read operations which is related to all are
Data are read from other group systems as the group system where the task, that is, the group system for obtaining data is identical
, the group system for providing data can be identical or different.For this purpose, in some embodiments of the present embodiment, in institute
The mark stated and can for example record the mark of the task in task scheduling daily record, obtain the group system of data, provides number
According to group system mark and the corresponding data volume of data read operation, wherein provide the mark of the group system of data with
There is correspondence between the corresponding data volume of data read operation.For example, it is assumed that needing during the task execution
One group system reads the first data volume from the second group system, in the task scheduling daily record of the task, obtains data
Group system the mark for being identified as the first group system, second group system of being identified as of the group system of data is provided
Mark, the corresponding data volume of data read operation are the first data volume, wherein the mark of the second group system and the first data volume
Between have correspondence.For another example, it is assumed that in addition to needing the first group system from the second cluster during the task execution
It is read in system except the first data volume, it is also necessary to which the first group system reads the second data volume from third group system, then
In the task scheduling daily record of the task in addition to record have the mark of the task, the mark of the first group system and with pair
Except the mark and the first data volume of the second group system that should be related to, the mark and of third group system can also be recorded
Two data volumes, wherein the mark of third group system and the second data volume have correspondence.
In some embodiments of the present embodiment, the mode that task submits equipment to generate task scheduling daily record is for example specific
Can be：Task submits equipment to be instructed in response to the submission of task, obtains the input file and file size of task；Task is submitted
Equipment summarizes all input files of the task according to the group system for providing data, obtain the task across collection
Group's input information；Task submits equipment to generate task scheduling daily record according to across the cluster input information of the task.Wherein, task
Submit equipment that can obtain input file and file size, the format of input file information by the input file information of task
Such as can be<File format>://<Cluster>/<File path>/:<File size>.For example, if the input text of task
Part information is pangu://fake_cluster2/fake_dir2/fake_path:100, pangu indicate what file system used
Agreement, fake_cluster2 indicate the group system (group system for providing data) where file, read the path of file
It is fake_dir2/fake_path, 100 indicate file size (i.e. the corresponding data volume of data read operation).If task is defeated
It is [pangu to enter fileinfo://fake_cluster2/fake_dir/fake_path:100,pangu://fake_
cluster3/fake_dir3/fake_path:100], then task submit equipment summarize the task obtained later across collection
Group's input information can be { fake_cluster2:100,fake_cluster3:300}.The task scheduling day further generated
Will can be (TaskName：fake_task,running_cluster:fake_cluster,ClusterInputInfo:
{fake_cluster2:100,fake_cluster3:300 }), wherein fake_task is the mark of the task, fake_
It is institute that the mark of group systems of the cluster where the task, fake_cluster2 and fake_cluster3, which are two,
The mark that task provides the group system of data is stated, 100 indicate that fake_cluster is from fake_cluster2 in the task
The data volume of reading, 300 indicate the data volume that fake_cluster is read from fake_cluster3 in the task.
Step 202 is based on the task scheduling daily record, counts the network flow across cluster in the distributed file system
Amount, the network flow across cluster includes the network flow between first group system and second group system.
It is understood that may be embodied in institute based on the network flow across cluster that the task scheduling log statistic goes out
State the network flow between corresponding two group systems of any one data read operation during task execution.For example,
If the task indicates that the first group system reads the first data volume from second system cluster, the network flow across cluster
Amount may include the network flow between the first group system and the second group system.For another example, if the task is in addition to instruction the
One group system is read from second system cluster except the first data volume, also indicates the first group system from third group system
The second data volume of middle reading, then the network flow across cluster can also include the first group system and third group system it
Between network flow.For another example, if the task reads the first data in addition to the first group system of instruction from second system cluster
Amount and the first group system are read from third group system except the second data volume, also indicate the second group system from third
Third data volume is read in group system, then the network flow across cluster can also include the second group system and third collection
Network flow between group's system.
The statistical result of step 203, the output network flow.
Specifically, monitoring system may include message handling system and stream processing system.Message handling system is receiving
When task submits the content that equipment is sent, task scheduling daily record can be identified from these contents and sends out task scheduling daily record
Give stream processing system, wherein know otherwise such as can be keyword filtering.Stream processing system is receiving at message
When the task scheduling daily record that reason system is sent, mark, the institute of the task can be obtained by parsing the task scheduling daily record
The information such as across company-data read operation and its corresponding data volume involved in time that the task of stating is acquired, the task.
Based on acquired information, stream processing system can be with the network flow across cluster in statistical cloth file system, in order to defeated
Go out to technical staff, to prompt the occupancy situation of network bandwidth in technical staff's distributed file system in advance.
It is understood that distributed file system would generally appoint when providing the services such as off-line calculation to one or more
Business submits device request to submit multiple tasks, and it can be that multiple tasks generate task meter respectively that one or more tasks, which submit equipment,
It draws daily record and is sent to monitoring system, monitoring system can be based on the task scheduling log statistic distributed field system of multiple tasks
Across the network flow of cluster in system.
In the present embodiment, various ways may be used to distribution in task based access control plan daily record, monitoring system
Network flow in file system across cluster is counted, to obtain a variety of different statistical results.
For example, in some embodiments of this implementation, monitoring system can count all tasks whithin a period of time and relate to
And two specified cluster systems between overall network flow, so as to prompt in advance between technical staff's specified cluster system
Network bandwidth occupancy situation.Specifically, step 202 for example may include：Institute is identified from the task scheduling daily record
State the first data volume；According to first data volume, in statistics specified time first described in the distributed file system
Network flow between group system and second group system；Wherein, the acquisition time of the task scheduling daily record belongs to
In the specified time.More specifically, monitoring system can find out the task meter of all tasks got in specified time
Daily record is drawn, identifies that all the second group systems of first group system pair read data from the task scheduling daily record of these tasks
Data volume, the sum of data volume identified is the within a specified time net between the first group system and the second group system
Network flow.
For another example, in other embodiments of the present embodiment, monitoring system can count the institute involved by particular task
There is the overall network flow that across company-data read operation occupies, it is total when so as to prompt technical staff's particular task to execute in advance
The network flow occupied altogether.That is, step 202 for example may include：Based on the task scheduling daily record, counts the task and refer to
The total amount of data read across group system shown, as the overall network flow involved in the task execution process.Further and
Speech, monitoring system can count the network flow occupied in total when all task executions, and it is maximum to occupy network flow for then output
One or more tasks, to which prompt technical staff excessively occupies the task of network bandwidth in advance, so that technical staff can
These tasks are handled in advance, it is excessive to avoid network bandwidth consumption.
It is understood that the statistical result of network flow is checked at any time for the ease of technical staff, in the present embodiment
In some embodiments, after monitoring system obtains the statistical result of network flow, other than being exported, such as can be with
The statistical result of the network flow is saved in data persistence system.Wherein, data persistence system for example can be several
According to library system.Certainly, the task scheduling daily record got can be also saved in data persistence system by monitoring system, so as to full
The subsequent statistical demand of foot.
It should be noted that in order to enable a technician to notice that the network bandwidth being subsequently likely to occur is different
MP1604840 is normal, and in some embodiments of the present embodiment, monitoring system for example may also respond to the system
Meter result meets the prerequisite for indicating exception of network traffic, exports warning message.For example, when if statistical result is specified
Overall network flow in interior distributed file system between the first group system and the second group system, the then prerequisite
It can be the network flow upper limit between group system, if statistical result is more than the prerequisite, export warning message.Again
Such as, if statistical result is the network flow occupied in total when particular task execution, the prerequisite can be that task occupies
The network flow upper limit, if statistical result be more than the prerequisite, export warning message.
In a realization Sample Scenario of the present embodiment, the mark of task for example can be 001, where task run
The mark of group system can be for example AY-A, and the input file information of task for example may include the following contents：
pangu://AY-A/fake_dir2/fake_path:45678
pangu://AY-B/fake_dir/fake_path:123456
pangu://AY-C/fake_dir2/fake_path:234567
pangu://AY-D/fake_dir2/fake_path:345678
The task scheduling daily record of the task for example may include the following contents：
TaskID：001
Cluster：AY-A
ClusterInputInfo：{“AY-B”:123456,“AY-C”:234567,“AY-D”:345678}
Monitoring system parses the task scheduling daily record of the task, and obtained information for example can be shown in table 1
Content：
Table 1
| This cluster | Remote cluster | TaskID | Size |
| AY-A | AY-B | 001 | 123456 |
| AY-A | AY-C | 001 | 234567 |
| AY-A | AY-D | 001 | 345678 |
Monitoring system for example can be content shown in table 2 based on the statistical result that the task scheduling log statistic obtains：
Table 2
Technical solution through this embodiment, it is contemplated that across company-data read operation is the behaviour in task implementation procedure
Make, and across company-data read operation has not been performed when task is submitted, the corresponding network flow of across company-data read operation
Amount does not occur in distributed file system yet, can be that task generates task scheduling daily record when task is submitted therefore,
Therefore the corresponding network flow of across company-data operation that task instruction can be recorded in task scheduling daily record is held in task
Before row, the task scheduling daily record based on each task can count the network flow in distributed file system across cluster in advance
Measure and export, therefore, distributed file system provide can be predicted when the services such as off-line calculation and export will execute across collection
Group's required network flow of data read operation, to prompt largely across company-data reading in distributed file system in advance
Operation occupies the case where a large amount of network bandwidths, is largely occupied by across company-data read operation to avoid network bandwidth, to keep away
The normal operation for exempting from distributed file system is affected.
Referring to Fig. 3, a kind of flow diagram of the method for monitoring traffic in network in the embodiment of the present application is shown.In this reality
It applies in example, the method is applied to task and equipment, i.e. the method is submitted to submit equipment to execute by the task, and the task carries
The task that equipment is used to handle in distributed file system is handed over to submit.The method for example can specifically include following steps：
Step 301, the submission instruction in response to task, task scheduling daily record is generated for the task；
Step 302 sends the task scheduling daily record to monitoring system, so that the monitoring system is based on the task meter
Daily record is drawn, counts the network flow across cluster in the distributed file system, and export the statistical result of the network flow；
Wherein, the task scheduling daily record includes that the task indicates that the first group system is read from the second group system
The first data volume, the network flow across cluster includes between first group system and second group system
Network flow.
Optionally, the task scheduling daily record for example can also include that the task indicates first group system from the
The third that the second data volume and second group system read in three group systems is read from the third group system
Data volume, the network flow across cluster further include the network between first group system and the third group system
Network flow between flow and second group system and the third group system.
Optionally, the identifying of the task, first cluster can be for example recorded in the task scheduling daily record
The identifying of system, the mark of second group system and first data volume, wherein the mark of second group system
There is correspondence with first data volume.
Technical solution through this embodiment, it is contemplated that across company-data read operation is the behaviour in task implementation procedure
Make, and across company-data read operation has not been performed when task is submitted, the corresponding network flow of across company-data read operation
Amount does not occur in distributed file system yet, can be that task generates task scheduling daily record when task is submitted therefore,
Therefore the corresponding network flow of across company-data operation that task instruction can be recorded in task scheduling daily record is held in task
Before row, the task scheduling daily record based on each task can count the network flow in distributed file system across cluster in advance
Measure and export, therefore, distributed file system provide can be predicted when the services such as off-line calculation and export will execute across collection
Group's required network flow of data read operation, to prompt largely across company-data reading in distributed file system in advance
Operation occupies the case where a large amount of network bandwidths, is largely occupied by across company-data read operation to avoid network bandwidth, to keep away
The normal operation for exempting from distributed file system is affected.
Referring to Fig. 4, a kind of structural schematic diagram of the device of monitoring traffic in network in the embodiment of the present application is shown.In this reality
It applies in example, described device is configured at monitoring system.Described device for example can specifically include：
Acquiring unit 401, the task scheduling for submitting equipment to be generated for the task when task is submitted for obtaining task
Daily record, the task that the task submits equipment to be used to handle in distributed file system are submitted, and the task scheduling daily record includes
The task indicates the first data volume that the first group system is read from the second group system；
Statistic unit 402 counts in the distributed file system for being based on the task scheduling daily record across cluster
Network flow, the network flow across cluster include the network between first group system and second group system
Flow；
First output unit 403, the statistical result for exporting the network flow.
Optionally, the task scheduling daily record for example can also include that the task indicates first group system from the
The third that the second data volume and second group system read in three group systems is read from the third group system
Data volume, the network flow across cluster further include the network between first group system and the third group system
Network flow between flow and second group system and the third group system.
Optionally, the statistic unit 402, such as can be specifically used for：
First data volume is identified from the task scheduling daily record；
According to first data volume, the first cluster system described in the distributed file system in statistics specified time
Network flow between system and second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
Optionally, the statistic unit 402, such as can be specifically used for：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as
Overall network flow involved in the task execution process.
Optionally, described device for example can also include：
Storage unit, for the statistical result of the network flow to be saved in data persistence system.
Optionally, described device for example can also include：
Second output unit exports warning message for meeting prerequisite in response to the statistical result.
Optionally, the identifying of the task, first cluster can be for example recorded in the task scheduling daily record
The identifying of system, the mark of second group system and first data volume, wherein the mark of second group system
There is correspondence with first data volume.
Technical solution through this embodiment, it is contemplated that across company-data read operation is the behaviour in task implementation procedure
Make, and across company-data read operation has not been performed when task is submitted, the corresponding network flow of across company-data read operation
Amount does not occur in distributed file system yet, can be that task generates task scheduling daily record when task is submitted therefore,
Therefore the corresponding network flow of across company-data operation that task instruction can be recorded in task scheduling daily record is held in task
Before row, the task scheduling daily record based on each task can count the network flow in distributed file system across cluster in advance
Measure and export, therefore, distributed file system provide can be predicted when the services such as off-line calculation and export will execute across collection
Group's required network flow of data read operation, to prompt largely across company-data reading in distributed file system in advance
Operation occupies the case where a large amount of network bandwidths, is largely occupied by across company-data read operation to avoid network bandwidth, to keep away
The normal operation for exempting from distributed file system is affected.
Referring to Fig. 5, a kind of structural schematic diagram of the device of monitoring traffic in network in the embodiment of the present application is shown.In this reality
It applies in example, described device is configured at task and equipment, the task is submitted to submit equipment for handling in distributed file system
Task is submitted.
Described device for example can specifically include：
Generation unit 501, instructs for the submission in response to task, and task scheduling daily record is generated for the task；
Transmission unit 502, for sending the task scheduling daily record to monitoring system, so that the monitoring system is based on institute
Task scheduling daily record is stated, counts the network flow across cluster in the distributed file system, and export the network flow
Statistical result, the network flow across cluster include the network between first group system and second group system
Flow；
Wherein, the task scheduling daily record includes that the task indicates that the first group system is read from the second group system
The first data volume.
Optionally, the task scheduling daily record for example can also include that the task indicates first group system from the
The third that the second data volume and second group system read in three group systems is read from the third group system
Data volume, the network flow across cluster further include the network between first group system and the third group system
Network flow between flow and second group system and the third group system.
Optionally, the identifying of the task, first cluster can be for example recorded in the task scheduling daily record
The identifying of system, the mark of second group system and first data volume, wherein the mark of second group system
There is correspondence with first data volume.
Technical solution through this embodiment, it is contemplated that across company-data read operation is the behaviour in task implementation procedure
Make, and across company-data read operation has not been performed when task is submitted, the corresponding network flow of across company-data read operation
Amount does not occur in distributed file system yet, can be that task generates task scheduling daily record when task is submitted therefore,
Therefore the corresponding network flow of across company-data operation that task instruction can be recorded in task scheduling daily record is held in task
Before row, the task scheduling daily record based on each task can count the network flow in distributed file system across cluster in advance
Measure and export, therefore, distributed file system provide can be predicted when the services such as off-line calculation and export will execute across collection
Group's required network flow of data read operation, to prompt largely across company-data reading in distributed file system in advance
Operation occupies the case where a large amount of network bandwidths, is largely occupied by across company-data read operation to avoid network bandwidth, to keep away
The normal operation for exempting from distributed file system is affected.
Referring to Fig. 6, a kind of structural schematic diagram of the system of monitoring traffic in network in the embodiment of the present application is shown.In this reality
It applies in example, the system of the monitoring traffic in network for example may include monitoring system 601, task submission equipment 602 and distribution
File system 603；The task that the task submits equipment 602 to be used to handle in distributed file system 603 is submitted；The distribution
Formula file system 603 includes at least the first group system 604 and the second group system 605；
The task submits equipment 602, is instructed for the submission in response to task, and task scheduling day is generated for the task
Will；Wherein, the task scheduling daily record includes that the task indicates that the first group system 604 is read from the second group system 605
The first data volume taken；
The monitoring system 601 submits equipment 602 to be generated for the task when task is submitted for obtaining the task
Task scheduling daily record, be based on the task scheduling daily record, count the network flow across cluster in the distributed file system 603
Amount, exports the statistical result of the network flow；Wherein, the network flow across cluster includes first group system
Network flow between 604 and second group system 605.
Optionally, the distributed file system further includes third group system 606；
The task scheduling daily record further includes that the task indicates first group system 604 from third group system
The third number that the second data volume and second group system 605 read in 606 is read from the third group system 606
According to amount, the network flow across cluster further includes between first group system 604 and the third group system 606
Network flow between network flow and second group system 605 and the third group system 606.
Optionally, in order to count the network flow in the distributed file system 603 across cluster, the monitoring system
601 are specifically used for：
First data volume is identified from the task scheduling daily record；
According to first data volume, first collection described in the distributed file system 603 in statistics specified time
Network flow between group's system 604 and second group system 605；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
Optionally, in order to count the network flow in the distributed file system 603 across cluster, the monitoring system
601 are specifically used for：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as
Overall network flow involved in the task execution process.
Optionally, the monitoring system 601 is additionally operable to：
The statistical result of the network flow is saved in data persistence system 607.
Optionally, the monitoring system 601 is additionally operable to：
Meet prerequisite in response to the statistical result, exports warning message.
Optionally, record has the identifying of the task, first group system 604 in the task scheduling daily record
Mark, the mark of second group system 605 and first data volume, wherein the mark of second group system 605
There is correspondence with first data volume.
Technical solution through this embodiment, it is contemplated that across company-data read operation is the behaviour in task implementation procedure
Make, and across company-data read operation has not been performed when task is submitted, the corresponding network flow of across company-data read operation
Amount does not occur in distributed file system yet, can be that task generates task scheduling daily record when task is submitted therefore,
Therefore the corresponding network flow of across company-data operation that task instruction can be recorded in task scheduling daily record is held in task
Before row, the task scheduling daily record based on each task can count the network flow in distributed file system across cluster in advance
Measure and export, therefore, distributed file system provide can be predicted when the services such as off-line calculation and export will execute across collection
Group's required network flow of data read operation, to prompt largely across company-data reading in distributed file system in advance
Operation occupies the case where a large amount of network bandwidths, is largely occupied by across company-data read operation to avoid network bandwidth, to keep away
The normal operation for exempting from distributed file system is affected.
It should be noted that herein, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.The terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or equipment including a series of elements includes not only those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or equipment institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including institute
State in the process, method, article or equipment of element that there is also other identical elements.
For device embodiments, since it corresponds essentially to embodiment of the method, so related place is referring to method reality
Apply the part explanation of example.Apparatus embodiments described above are merely indicative, wherein described be used as separating component
The unit of explanation may or may not be physically separated, and the component shown as unit can be or can also
It is not physical unit, you can be located at a place, or may be distributed over multiple network units.It can be according to actual
It needs that some or all of module therein is selected to achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not
In the case of making the creative labor, you can to understand and implement.
The above is only the specific implementation mode of the application, it is noted that for the ordinary skill people of the art
For member, under the premise of not departing from the application principle, several improvements and modifications can also be made, these improvements and modifications are also answered
It is considered as the protection domain of the application.
Claims (19)
1. a kind of method of monitoring traffic in network, which is characterized in that it is applied to monitoring system, including：
Acquisition task submits the task scheduling daily record that equipment is generated when task is submitted for the task, the task to submit equipment
It is submitted for handling the task in distributed file system, the task scheduling daily record includes that the task indicates the first cluster system
The first data volume that system is read from the second group system；
Based on the task scheduling daily record, the network flow across cluster in the distributed file system is counted, it is described across cluster
Network flow include network flow between first group system and second group system；
Export the statistical result of the network flow.
2. according to the method described in claim 1, it is characterized in that, the task scheduling daily record further includes task instruction institute
The second data volume that the first group system is read from third group system and second group system are stated from the third collection
The third data volume read in group's system, the network flow across cluster further includes first group system and the third
The network flow between network flow and second group system and the third group system between group system.
3. according to the method described in claim 1, it is characterized in that, described be based on the task scheduling daily record, described point of statistics
Across the network flow of cluster in cloth file system, including：
First data volume is identified from the task scheduling daily record；
According to first data volume, in statistics specified time the first group system described in the distributed file system with
Network flow between second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
4. according to the method described in claim 1, it is characterized in that, described be based on the task scheduling daily record, described point of statistics
Across the network flow of cluster in cloth file system, specially：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as described
Overall network flow involved in task execution process.
5. according to the method described in claim 1, it is characterized in that, further including：
The statistical result of the network flow is saved in data persistence system.
6. according to the method described in claim 1, it is characterized in that, further including：
Meet prerequisite in response to the statistical result, exports warning message.
7. according to the method described in claim 1, it is characterized in that, record has the task in the task scheduling daily record
Mark, the identifying of first group system, the mark of second group system and first data volume, wherein described
The mark of second group system has correspondence with first data volume.
8. a kind of method of monitoring traffic in network, which is characterized in that be applied to task and submit equipment, the task that equipment is submitted to use
It is submitted in handling the task in distributed file system；
The method includes：
In response to the submission instruction of task, task scheduling daily record is generated for the task；
The task scheduling daily record is sent to monitoring system, so that the monitoring system is based on the task scheduling daily record, statistics
Across the network flow of cluster in the distributed file system, and export the statistical result of the network flow；
Wherein, the task scheduling daily record include that the task indicates that the first group system reads from the second group system the
One data volume, the network flow across cluster include the network between first group system and second group system
Flow.
9. according to the method described in claim 8, it is characterized in that, the task scheduling daily record further includes task instruction institute
The second data volume that the first group system is read from third group system and second group system are stated from the third collection
The third data volume read in group's system, the network flow across cluster further includes first group system and the third
The network flow between network flow and second group system and the third group system between group system.
10. according to the method described in claim 8, it is characterized in that, record has the task in the task scheduling daily record
Identify, the identifying of first group system, the mark of second group system and first data volume, wherein institute
The mark and first data volume for stating the second group system have correspondence.
11. a kind of device of monitoring traffic in network, which is characterized in that it is configured at monitoring system, including：
Acquiring unit, the task scheduling daily record for submitting equipment to be generated for the task when task is submitted for obtaining task, institute
It states the task that task submits equipment to be used to handle in distributed file system to submit, the task scheduling daily record includes the task
Indicate the first data volume that the first group system is read from the second group system；
Statistic unit counts the network flow across cluster in the distributed file system for being based on the task scheduling daily record
Amount, the network flow across cluster includes the network flow between first group system and second group system；
First output unit, the statistical result for exporting the network flow.
12. a kind of device of monitoring traffic in network, which is characterized in that the task that is configured at submits equipment, the task to submit equipment
It is submitted for handling the task in distributed file system；
Described device includes：
Generation unit, instructs for the submission in response to task, and task scheduling daily record is generated for the task；
Transmission unit, for sending the task scheduling daily record to monitoring system, so that the monitoring system is based on the task
Plan daily record, counts the network flow across cluster in the distributed file system, and export the statistics knot of the network flow
Fruit；
Wherein, the task scheduling daily record include that the task indicates that the first group system reads from the second group system the
One data volume, the network flow across cluster include the network between first group system and second group system
Flow.
13. a kind of system of monitoring traffic in network, which is characterized in that including：Monitoring system, task submit equipment and distribution text
Part system；The task that the task submits equipment to be used to handle in distributed file system is submitted；The distributed file system
Including at least the first group system and the second group system；
The task submits equipment, is instructed for the submission in response to task, and task scheduling daily record is generated for the task；Its
In, the task scheduling daily record includes that the task indicates the first data that the first group system is read from the second group system
Amount；
The monitoring system, the task scheduling for submitting equipment to be generated for the task when task is submitted for obtaining the task
Daily record is based on the task scheduling daily record, counts the network flow across cluster in the distributed file system, export the net
The statistical result of network flow；Wherein, the network flow across cluster includes first group system and second cluster
Network flow between system.
14. system according to claim 13, which is characterized in that the distributed file system further includes third cluster system
System；
The task scheduling daily record further includes that the task indicates that first group system is read from third group system
The third data volume that second data volume and second group system are read from the third group system, it is described across cluster
Network flow further includes the network flow and second collection between first group system and the third group system
Network flow between group's system and the third group system.
15. system according to claim 13, which is characterized in that in order to count in the distributed file system across cluster
Network flow, the monitoring system is specifically used for：
First data volume is identified from the task scheduling daily record；
According to first data volume, in statistics specified time the first group system described in the distributed file system with
Network flow between second group system；
Wherein, the acquisition time of the task scheduling daily record belonged in the specified time.
16. system according to claim 13, which is characterized in that in order to count in the distributed file system across cluster
Network flow, the monitoring system is specifically used for：
Based on the task scheduling daily record, the total amount of data of the task instruction read across group system is counted, as described
Overall network flow involved in task execution process.
17. system according to claim 13, which is characterized in that the monitoring system is additionally operable to：
The statistical result of the network flow is saved in data persistence system.
18. system according to claim 13, which is characterized in that the monitoring system is additionally operable to：
Meet prerequisite in response to the statistical result, exports warning message.
19. system according to claim 13, which is characterized in that record has the task in the task scheduling daily record
Identify, the identifying of first group system, the mark of second group system and first data volume, wherein institute
The mark and first data volume for stating the second group system have correspondence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710007945.XA CN108282378B (en) | 2017-01-05 | 2017-01-05 | Method and device for monitoring network flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710007945.XA CN108282378B (en) | 2017-01-05 | 2017-01-05 | Method and device for monitoring network flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108282378A true CN108282378A (en) | 2018-07-13 |
| CN108282378B CN108282378B (en) | 2021-11-09 |
Family
ID=62800623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710007945.XA Active CN108282378B (en) | 2017-01-05 | 2017-01-05 | Method and device for monitoring network flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108282378B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102014137A (en) * | 2010-12-13 | 2011-04-13 | 哈尔滨工业大学 | Common distributed data recording device and method based on HLA (high level architecture) |
| CN102855299A (en) * | 2012-08-16 | 2013-01-02 | 上海引跑信息科技有限公司 | Method for realizing iterative migration of distributed database without interrupting service |
| CN103327048A (en) * | 2012-03-22 | 2013-09-25 | 阿里巴巴集团控股有限公司 | Online-application data matching method and device |
| CN103389946A (en) * | 2013-07-16 | 2013-11-13 | 中国科学院计算技术研究所 | Fragmentization removal method and system |
| CN103455633A (en) * | 2013-09-24 | 2013-12-18 | 浪潮齐鲁软件产业有限公司 | Method of distributed analysis for massive network detailed invoice data |
| CN103678520A (en) * | 2013-11-29 | 2014-03-26 | 中国科学院计算技术研究所 | Multi-dimensional interval query method and system based on cloud computing |
| CN106034160A (en) * | 2015-03-19 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Distributed computing system and method |
| US20160330281A1 (en) * | 2015-05-07 | 2016-11-10 | Dell Products L.P. | Systems and methods to improve read/write performance in object storage applications |
-
2017
- 2017-01-05 CN CN201710007945.XA patent/CN108282378B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102014137A (en) * | 2010-12-13 | 2011-04-13 | 哈尔滨工业大学 | Common distributed data recording device and method based on HLA (high level architecture) |
| CN103327048A (en) * | 2012-03-22 | 2013-09-25 | 阿里巴巴集团控股有限公司 | Online-application data matching method and device |
| CN102855299A (en) * | 2012-08-16 | 2013-01-02 | 上海引跑信息科技有限公司 | Method for realizing iterative migration of distributed database without interrupting service |
| CN103389946A (en) * | 2013-07-16 | 2013-11-13 | 中国科学院计算技术研究所 | Fragmentization removal method and system |
| CN103455633A (en) * | 2013-09-24 | 2013-12-18 | 浪潮齐鲁软件产业有限公司 | Method of distributed analysis for massive network detailed invoice data |
| CN103678520A (en) * | 2013-11-29 | 2014-03-26 | 中国科学院计算技术研究所 | Multi-dimensional interval query method and system based on cloud computing |
| CN106034160A (en) * | 2015-03-19 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Distributed computing system and method |
| US20160330281A1 (en) * | 2015-05-07 | 2016-11-10 | Dell Products L.P. | Systems and methods to improve read/write performance in object storage applications |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108282378B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104462121B (en) | Data processing method, apparatus and system | |
| CN105488610B (en) | Fault real-time analysis and diagnosis method for power application system | |
| EP2895956B1 (en) | Method and system for monitoring execution of user request in distributed system | |
| CN109471845A (en) | Blog management method, server and computer readable storage medium | |
| CN109284251A (en) | Blog management method, device, computer equipment and storage medium | |
| CN103401698B (en) | For the monitoring system that server health is reported to the police in server set group operatione | |
| CN108989136A (en) | Business end to end performance monitoring method and device | |
| CN105302697B (en) | A kind of running state monitoring method and system of density data model database | |
| CN109039819A (en) | Delay statistics method, apparatus, system and storage medium | |
| CN103067218A (en) | High speed network data package content analysis device | |
| CN113505044B (en) | Database warning method, device, equipment and storage medium | |
| CN109117341A (en) | A kind of monitoring method of virtual machine, device, equipment and medium | |
| CN106992886A (en) | A kind of log analysis method and device based on distributed storage | |
| CN106254137A (en) | The alarm root-cause analysis system and method for supervisory systems | |
| CN103870549B (en) | The method for cleaning and device of petroleum geology software data | |
| CN106034047B (en) | Data processing method and device | |
| CN106789158A (en) | Damage identification method and system are insured in a kind of cloud service | |
| CN108234189B (en) | Alarm data processing method and device | |
| CN109819019A (en) | Monitoring and statistical analysis technique and system for the acquisition of large scale network data | |
| CN109818808A (en) | Method for diagnosing faults, device and electronic equipment | |
| US10289520B2 (en) | Adaptive application performance analysis | |
| CN104038388B (en) | Based on distributed Internet of Things Auto-Test System and method of testing | |
| CN108108289A (en) | A kind of cluster resource statistical method, system, device and readable storage system | |
| CN108494635A (en) | A kind of network flow detection system based on cloud computing | |
| CN102055620A (en) | Method and system for monitoring user experience |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |