CN110266768A - A kind of data transmission method and system - Google Patents
A kind of data transmission method and system Download PDFInfo
- Publication number
- CN110266768A CN110266768A CN201910442006.7A CN201910442006A CN110266768A CN 110266768 A CN110266768 A CN 110266768A CN 201910442006 A CN201910442006 A CN 201910442006A CN 110266768 A CN110266768 A CN 110266768A
- Authority
- CN
- China
- Prior art keywords
- data
- network management
- monitoring network
- networking monitoring
- scheduling system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
The embodiment of the invention provides data transmission methods and system.One specific embodiment of this method, which includes: higher level, determines at least one target matrix depending on networking monitoring network management scheduling system, and the corresponding relationship based on tables of data and partition identification determines the corresponding partition identification of each target matrix；Data read command is sent to junior's view networking monitoring network management scheduling system；Receive the delta data read out from the file for corresponding to partition identification that junior sends depending on networking monitoring network management scheduling system.On the one hand, delta data is obtained from the file that the delta data of interested tables of data is stored in, without obtaining the delta data of all tables of data when obtaining delta data each time, reduce the transmission that higher level regards networking monitoring network management scheduling system and junior's view networking monitoring network management scheduling unnecessary delta data of system.On the other hand, the management for facilitating delta data promotes the flexibility of business processing.
Description
Technical field
This application involves computer fields, and in particular to view networking arenas more particularly to data transmission method and system.
Background technique
The real-time network networked to can be realized HD video real-time Transmission is regarded, pushing numerous Internet applications to high definition view
Frequencyization, high definition are face-to-face.It is used to carry out the operating condition of the equipment in view networking depending on networking monitoring network management scheduling system
Monitoring, the equipment in management view networking, the resource in scheduling view networking.It is multiple depending on having in networking monitoring network management scheduling system
There is the higher level of cascade connection to regard networking monitoring network management scheduling system and junior's view networking monitoring network management scheduling system warp
Often there is the demand of transmission data, higher level regards networking monitoring network management scheduling system and junior's view networking monitoring network management scheduling
The speed of system transmission data directly affects monitoring efficiency, the efficiency of management, dispatching efficiency.
Summary of the invention
In view of the above problems, it proposes the embodiment of the present invention and overcomes the above problem or at least partly in order to provide one kind
A kind of data transmission method for solving the above problems, a kind of data transmission system.
To solve the above-mentioned problems, the embodiment of the invention provides a kind of data transmission methods, comprising: higher level regards networking prison
Control network management scheduling system determines at least one target matrix, and the corresponding relationship based on tables of data and partition identification,
Determine the corresponding partition identification of each of at least one target matrix target matrix；Higher level regards networking monitoring networking pipe
It manages scheduling system and sends data read command to junior's view networking monitoring network management scheduling system, data read command includes:
The corresponding partition identification of described each target matrix, wherein the higher level regard networking monitoring network management scheduling system with
Junior's view networking monitoring network management scheduling system passes through view networking transport data；It is corresponding for each target matrix
Partition identification, the junior depending on networking monitoring network management dispatch system from correspond to the partition identification file in read
Take out delta data, wherein the delta data of each tables of data dispatches system by junior's view networking monitoring network management respectively
It is pre-written into corresponding file, the corresponding file of each tables of data respectively corresponds a partition identification；On
Grade receives the reading that junior's view networking monitoring network management scheduling system is sent depending on networking monitoring network management scheduling system
Delta data out.
The embodiment of the invention also provides a kind of data transmission systems, comprising: higher level regards the scheduling of networking monitoring network management
System, junior's view networking monitoring network management dispatch system, the higher level regard networking monitoring network management scheduling system with it is described
Junior regards networking monitoring network management scheduling system and passes through view networking transport data；Higher level regards networking monitoring network management scheduling system
System is configured as: being determined at least one target matrix, and the corresponding relationship based on tables of data and partition identification, is determined at least
The corresponding partition identification of each of one target matrix target matrix；To junior's view networking monitoring network management scheduling
System sends data read command, and data read command includes: the corresponding partition identification of each described target matrix；It receives
The delta data read out that the junior sends depending on networking monitoring network management scheduling system；Junior regards networking monitoring networking pipe
Reason scheduling system is configured as: partition identification corresponding for each target matrix, from corresponding to the partition identification
Delta data is read out in file, wherein the delta data of each target matrix is respectively by junior's view networking monitoring connection
Net management scheduling system is pre-written into corresponding file, and the corresponding file of each target matrix is right respectively
Answer a partition identification.
The embodiment of the present invention includes following advantages:
The tables of data of each tables of data is respectively stored in different points depending on networking monitoring network management scheduling system by junior
The tables of data of each tables of data is respectively stored in different files depending on networking monitoring network management scheduling system by Qu Zhong, i.e. junior
In folder.To, on the one hand, higher level regards networking monitoring network management scheduling system according to demand, obtains from interested subregion
Data obtain delta data from the file that the delta data of interested tables of data is stored in, without each
The delta data of all tables of data is obtained when secondary acquisition delta data, thus, reduce higher level and regards networking monitoring network management
The transmission of scheduling system and junior's view networking monitoring network management scheduling unnecessary delta data of system, improves delta data
Read or write speed.On the other hand, the delta data of all tables of data is distributed to multiple subregions to store respectively, i.e., multiple
It is stored respectively in file, facilitates the management of delta data, higher level can root depending on networking monitoring network management scheduling system
According to demand, the delta data in any subregion is obtained, i.e., from the file that the delta data of any data table memory is stored in
The delta data needed is obtained, the flexibility of business processing is promoted.
Detailed description of the invention
Fig. 1 is a kind of networking schematic diagram of view networking of the invention；
Fig. 2 is a kind of hardware structural diagram of node server of the invention；
Fig. 3 is a kind of hardware structural diagram of access switch of the invention；
Fig. 4 is the hardware structural diagram that a kind of Ethernet association of the invention turns gateway；
Fig. 5 is a kind of flow chart of data transmission method provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of data transmission system provided in an embodiment of the present invention.
Specific embodiment
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real
Applying mode, the present invention is described in further detail.
It is the important milestone of network Development depending on networking, is a real-time network, can be realized HD video real-time Transmission,
Push numerous Internet applications to HD video, high definition is face-to-face.
Real-time high-definition video switching technology is used depending on networking, it can be such as high in a network platform by required service
Clear video conference, Intellectualized monitoring analysis, emergency command, digital broadcast television, delay TV, the Web-based instruction, shows video monitoring
Field live streaming, VOD program request, TV Mail, individual character records (PVR), Intranet (manages) channel by oneself, intelligent video Broadcast Control, information publication
All be incorporated into a system platform etc. services such as tens of kinds of videos, voice, picture, text, communication, data, by TV or
Computer realizes that high-definition quality video plays.
Embodiment in order to enable those skilled in the art to better understand the present invention is introduced to depending on networking below:
Depending on networking, applied portion of techniques is as described below:
Network technology (Network Technology)
Traditional ethernet (Ethernet) is improved depending on the network technology innovation networked, with potential huge on network
First video flow.It is exchanged different from simple network packet packet switch (Packet Switching) or lattice network
(Circuit Switching) meets Streaming demand using Packet Switching depending on networking technology.Depending on skill of networking
Art has flexible, the simple and low price of packet switch, is provided simultaneously with the quality and safety assurance of circuit switching, realizes the whole network friendship
Change the seamless connection of formula virtual circuit and data format.
Switching technology (Switching Technology)
Two advantages of asynchronous and packet switch that Ethernet is used depending on networking eliminate Ethernet under the premise of complete compatible and lack
It falls into, has the end-to-end seamless connection of the whole network, direct user terminal, directly carrying IP data packet.User data is in network-wide basis
It is not required to any format conversion.It is the more advanced form of Ethernet depending on networking, is a real-time exchange platform, can be realized at present mutually
The whole network large-scale high-definition realtime video transmission that networking cannot achieve pushes numerous network video applications to high Qinghua, unitizes.
Server technology (Server Technology)
It is different from traditional server, its Streaming Media depending on the server technology in networking and unified video platform
Transmission be built upon it is connection-oriented on the basis of, data-handling capacity is unrelated with flow, communication time, single network layer energy
Enough transmitted comprising signaling and data.For voice and video business, handled depending on networking and unified video platform Streaming Media
Complexity many simpler than data processing, efficiency substantially increase hundred times or more than traditional server.
Reservoir technology (Storage Technology)
The ultrahigh speed reservoir technology of unified video platform in order to adapt to the media content of vast capacity and super-flow and
Using state-of-the-art real time operating system, the programme information in server instruction is mapped to specific hard drive space, media
Content is no longer pass through server, and moment is directly delivered to user terminal, and user waits typical time less than 0.2 second.It optimizes
Sector distribution greatly reduces the mechanical movement of hard disc magnetic head tracking, and resource consumption only accounts for the 20% of the internet ad eundem IP, but
The concurrent flow greater than 3 times of traditional disk array is generated, overall efficiency promotes 10 times or more.
Network security technology (Network Security Technology)
Depending on the structural design networked by servicing independent licence system, equipment and the modes such as user data is completely isolated every time
The network security problem that puzzlement internet has thoroughly been eradicated from structure, does not need antivirus applet, firewall generally, has prevented black
The attack of visitor and virus, structural carefree secure network is provided for user.
It services innovative technology (Service Innovation Technology)
Business and transmission are fused together by unified video platform, whether single user, private user or a net
The sum total of network is all only primary automatic connection.User terminal, set-top box or PC are attached directly to unified video platform, obtain rich
The multimedia video service of rich colorful various forms.Unified video platform is traditional to substitute with table schema using " menu type "
Complicated applications programming, considerably less code, which can be used, can be realized complicated application, realize the new business innovation of " endless ".
Networking depending on networking is as described below:
It is a kind of central controlled network structure depending on networking, which can be Tree Network, Star network, ring network etc. class
Type, but centralized control node is needed to control whole network in network on this basis.
As shown in Figure 1, being divided into access net and Metropolitan Area Network (MAN) two parts depending on networking.
The equipment of access mesh portions can be mainly divided into 3 classes: node server, access switch, terminal (including various machines
Top box, encoding board, memory etc.).Node server is connected with access switch, and access switch can be with multiple terminal phases
Even, and it can connect Ethernet.
Wherein, node server is the node that centralized control functions are played in access net, can control access switch and terminal.
Node server can directly be connected with access switch, can also directly be connected with terminal.
Similar, the equipment of metropolitan area mesh portions can also be divided into 3 classes: metropolitan area server, node switch, node serve
Device.Metropolitan area server is connected with node switch, and node switch can be connected with multiple node servers.
Wherein, node server is the node server for accessing mesh portions, i.e. node server had both belonged to access wet end
Point, and belong to metropolitan area mesh portions.
Metropolitan area server is the node that centralized control functions are played in Metropolitan Area Network (MAN), can control node switch and node serve
Device.Metropolitan area server can be directly connected to node switch, can also be directly connected to node server.
It can be seen that be entirely a kind of central controlled network structure of layering depending on networking network, and node server and metropolitan area
The network controlled under server can be the various structures such as tree-shaped, star-like, cyclic annular.
Visually claim, access mesh portions can form unified video platform (part in virtual coil), and multiple unified videos are flat
Platform can form view networking；Each unified video platform can be interconnected by metropolitan area and wide area depending on networking.
Classify depending on networked devices
1.1 embodiment of the present invention can be mainly divided into 3 classes: server depending on the equipment in networking, interchanger (including ether
Net association turns gateway), terminal (including various set-top boxes, encoding board, memory etc.).Metropolitan Area Network (MAN) can be divided on the whole depending on networking
(or national net, World Wide Web etc.) and access net.
1.2 equipment for wherein accessing mesh portions can be mainly divided into 3 classes: node server, access switch (including ether
Net association turns gateway), terminal (including various set-top boxes, encoding board, memory etc.).
The specific hardware structure of each access network equipment are as follows:
Node server:
As shown in Fig. 2, mainly including Network Interface Module 201, switching engine module 202, CPU module 203, disk array
Module 204；
Wherein, Network Interface Module 201, the Bao Jun that CPU module 203, disk array module 204 are come in enter switching engine
Module 202；Switching engine module 202 look into the operation of address table 205 to the packet come in, to obtain the navigation information of packet；
And the packet is stored according to the navigation information of packet the queue of corresponding pack buffer 206；If the queue of pack buffer 206 is close
It is full, then it abandons；All pack buffer queues of 202 poll of switching engine module, are forwarded: 1) end if meeting the following conditions
It is less than that mouth sends caching；2) the queue package counting facility is greater than zero.Disk array module 204 mainly realizes the control to hard disk, packet
Include the operation such as initialization, read-write to hard disk；CPU module 203 be mainly responsible for access switch, terminal (not shown) it
Between protocol processes, to address table 205 (including descending protocol packet address table, uplink protocol package address table, data packet addressed table)
Configuration, and, the configuration to disk array module 204.
Access switch:
As shown in figure 3, mainly including Network Interface Module (downstream network interface module 301, uplink network interface module
302), switching engine module 303 and CPU module 304；
Wherein, the packet (upstream data) that downstream network interface module 301 is come in enters packet detection module 305；Packet detection mould
Whether mesh way address (DA), source address (SA), type of data packet and the packet length of the detection packet of block 305 meet the requirements, if met,
It then distributes corresponding flow identifier (stream-id), and enters switching engine module 303, otherwise abandon；Uplink network interface mould
The packet (downlink data) that block 302 is come in enters switching engine module 303；The data packet that CPU module 304 is come in enters switching engine
Module 303；Switching engine module 303 look into the operation of address table 306 to the packet come in, to obtain the navigation information of packet；
If the packet into switching engine module 303 is that downstream network interface is gone toward uplink network interface, in conjunction with flow identifier
(stream-id) packet is stored in the queue of corresponding pack buffer 307；If the queue of the pack buffer 307 is close full,
It abandons；If the packet into switching engine module 303 is not that downstream network interface is gone toward uplink network interface, according to packet
Navigation information is stored in the data packet queue of corresponding pack buffer 307；If the queue of the pack buffer 307 is close full,
Then abandon.
All pack buffer queues of 303 poll of switching engine module, may include two kinds of situations:
If the queue is that downstream network interface is gone toward uplink network interface, meets the following conditions and be forwarded: 1)
It is less than that the port sends caching；2) the queue package counting facility is greater than zero；3) token that rate control module generates is obtained；
If the queue is not that downstream network interface is gone toward uplink network interface, meets the following conditions and is forwarded:
1) it is less than to send caching for the port；2) the queue package counting facility is greater than zero.
Rate control module 308 is configured by CPU module 304, to all downlink networks in programmable interval
Interface generates token toward the pack buffer queue that uplink network interface is gone, to control the code rate of forwarded upstream.
CPU module 304 is mainly responsible for the protocol processes between node server, the configuration to address table 306, and,
Configuration to rate control module 308.
Ethernet association turns gateway:
As shown in figure 4, mainly including Network Interface Module (downstream network interface module 401, uplink network interface module
402), switching engine module 403, CPU module 404, packet detection module 405, rate control module 408, address table 406, Bao Huan
Storage 407 and MAC adding module 409, MAC removing module 410.
Wherein, the data packet that downstream network interface module 401 is come in enters packet detection module 405；Packet detection module 405 is examined
Ethernet mac DA, ethernet mac SA, Ethernet length or frame type, the view networking mesh way address of measured data packet
DA, whether meet the requirements depending on networking source address SA, depending on networking data Packet type and packet length, corresponding stream is distributed if meeting
Identifier (stream-id)；Then, MAC DA, MAC SA, length or frame type are subtracted by MAC removing module 410
(2byte), and enter corresponding receive and cache, otherwise abandon；
Downstream network interface module 401 detects the transmission caching of the port, according to the view of packet networking mesh if there is Bao Ze
Address D A knows the ethernet mac DA of corresponding terminal, adds the ethernet mac DA of terminal, Ethernet assists the MAC for turning gateway
SA, Ethernet length or frame type, and send.
The function that Ethernet association turns other modules in gateway is similar with access switch.
Terminal:
It mainly include Network Interface Module, Service Processing Module and CPU module；For example, set-top box mainly connects including network
Mouth mold block, video/audio encoding and decoding engine modules, CPU module；Encoding board mainly includes Network Interface Module, video encoding engine
Module, CPU module；Memory mainly includes Network Interface Module, CPU module and disk array module.
The equipment of 1.3 metropolitan area mesh portions can be mainly divided into 2 classes: node server, node switch, metropolitan area server.
Wherein, node switch mainly includes Network Interface Module, switching engine module and CPU module；Metropolitan area server mainly includes
Network Interface Module, switching engine module and CPU module are constituted.
2, networking data package definition is regarded
2.1 access network data package definitions
Access net data packet mainly include following sections: destination address (DA), source address (SA), reserve bytes,
payload(PDU)、CRC。
As shown in the table, the data packet for accessing net mainly includes following sections:
| DA | SA | Reserved | Payload | CRC |
Wherein:
Destination address (DA) is made of 8 bytes (byte), and first character section indicates type (such as the various associations of data packet
Discuss packet, multicast packet, unicast packet etc.), be up to 256 kinds of possibility, the second byte to the 6th byte is metropolitan area net address,
Seven, the 8th bytes are access net address；
Source address (SA) is also to be made of 8 bytes (byte), is defined identical as destination address (DA)；
Reserve bytes are made of 2 bytes；
The part payload has different length according to the type of different datagrams, is if it is various protocol packages
64 bytes are 32+1024=1056 bytes if it is single group unicast packets words, are not restricted to above 2 kinds certainly；
CRC is made of 4 bytes, and calculation method follows the Ethernet CRC algorithm of standard.
2.2 Metropolitan Area Network (MAN) packet definitions
The topology of Metropolitan Area Network (MAN) is pattern, may there is 2 kinds, connection even of more than two kinds, i.e. node switching between two equipment
It can all can exceed that 2 kinds between machine and node server, node switch and node switch, node switch and node server
Connection.But the metropolitan area net address of metropolitan area network equipment is uniquely, to close to accurately describe the connection between metropolitan area network equipment
System, introduces parameter in embodiments of the present invention: label, uniquely to describe a metropolitan area network equipment.
(Multi-Protocol Label Switch, multiprotocol label are handed over by the definition of label and MPLS in this specification
Change) label definition it is similar, it is assumed that between equipment A and equipment B there are two connection, then data packet from equipment A to equipment B just
There are 2 labels, data packet also there are 2 labels from equipment B to equipment A.Label is divided into label, outgoing label, it is assumed that data packet enters
The label (entering label) of equipment A is 0x0000, and the label (outgoing label) when this data packet leaves equipment A may reform into
0x0001.The networking process of Metropolitan Area Network (MAN) is to enter network process under centralized control, also means that address distribution, the label of Metropolitan Area Network (MAN)
Distribution be all to be dominated by metropolitan area server, node switch, node server be all passively execute, this point with
The label distribution of MPLS is different, and the distribution of the label of MPLS is the result that interchanger, server are negotiated mutually.
As shown in the table, the data packet of Metropolitan Area Network (MAN) mainly includes following sections:
| DA | SA | Reserved | Label | Payload | CRC |
That is destination address (DA), source address (SA), reserve bytes (Reserved), label, payload (PDU), CRC.Its
In, the format of label, which can refer to, such as gives a definition: label is 32bit, wherein high 16bit retains, only with low 16bit, its position
Set is between the reserve bytes and payload of data packet.
With reference to Fig. 5, a kind of flow chart of data transmission method provided in an embodiment of the present invention is shown, can specifically include
Following steps:
Step 501, higher level determines at least one target matrix depending on networking monitoring network management scheduling system, determines each
The corresponding partition identification of a target matrix.
In this application, depending on networking monitoring network management scheduling system be used for view networking in equipment operating condition into
Row monitoring, the equipment in management view networking, the resource in scheduling view networking.In order to view networking in equipment operating condition into
Row monitoring, the equipment in management view networking, the resource in scheduling view networking, need based on magnanimity with the equipment in view networking
Operating condition is monitored, manages and is analyzed depending on relevant data such as the equipment in networking, the resources in scheduling view networking,
Operating condition depending on the equipment in networking is monitored, manages the equipment in view networking, the resource in scheduling view networking.
In this application, higher level regards networking monitoring network management scheduling system, junior's view networking monitoring network management scheduling
System is simultaneously not specific to some view networking monitoring network management scheduling system.Higher level's view networking monitoring network management scheduling system,
Junior regards networking monitoring network management scheduling system and can dispatch as two view networking monitoring network managements with subordinate relation
System.Or two view networking monitoring network managements with incidence relation dispatch system.For example, in view networking
The operating condition of equipment is monitored, manages the equipment in view networking, in the resource process in scheduling view networking, a view networking
Monitor-Networking management scheduling system need to carry out another depending on the data generated in networking monitoring network management scheduling system into
The processing of one step, then this can be used as higher level depending on networking monitoring network management scheduling system and regards networking monitoring network management scheduling system
System, this another view networking monitoring network management scheduling system can be used as junior's view networking monitoring network management and dispatch system.
In this application, all tables of data can store in junior's view networking monitoring network management scheduling system.Data
The data stored in table can regard the equipment in networking, scheduling to be monitored, managing depending on the operating condition of the equipment in networking
Depending on the relevant data such as resource in networking.
In this application, target matrix is in all tables of data, and higher level regards networking monitoring network management scheduling system and needs
Determine the tables of data of its data situation.Higher level regards networking monitoring network management scheduling system by reading target matrix
Delta data, determine the data situation of target matrix.
In this application, the delta data of a tables of data indicates the variation feelings of changed data in the tables of data
Condition.For example, the delta data of a tables of data includes: the title of the changed field of field value in the tables of data, becomes
Field value etc. after change.
In this application, higher level regards networking monitoring network management scheduling system it needs to be determined that data in multiple tables of data
When situation of change, then each of multiple tables of data tables of data is used as target matrix.
In this application, higher level regard networking monitoring network management scheduling system read target matrix delta data it
Before, the change that a file carrys out data table memory can be distributed respectively in advance for each of all tables of data tables of data
Change data.Also referred to as the delta data subregion of all tables of data is stored.Each of all tables of data data
The delta data of table can be respectively written into the change of data table memory in advance by junior depending on networking monitoring network management scheduling system
In the file for changing data.
In this application, the region of the delta data of each data table memory can be referred to as a subregion.Each
The file that the delta data of tables of data is stored in respectively corresponds a partition identification.The variation of each data table data table
Pair that the delta data of the corresponding partition identification of the file that data are stored in and other data table data tables is stored in
The partition identification answered is different.
In this application, junior is depending on being stored with designation date table and partition identification in networking monitoring network management scheduling system
Corresponding relationship mapping table.The mapping table of the corresponding relationship of designation date table and partition identification, which can store, is matching
It sets in file or database.
In this application, junior can be in advance by designation date table and subregion mark depending on networking monitoring network management scheduling system
The mapping table of the corresponding relationship of knowledge is sent to higher level and regards networking monitoring network management scheduling system.
In this application, networking monitoring network management scheduling system can be regarded by higher level based on tables of data and partition identification
Corresponding relationship determines the corresponding partition identification of each target matrix.
In some embodiments, the file that the delta data of each tables of data is respectively stored in can be set first in advance
Then folder corresponds to the partition identification of file for each folder allocation one, it is thus possible to determine tables of data simultaneously
The file and subregion that the delta data of file, that is, tables of data corresponding with the corresponding relationship of partition identification, tables of data is stored in
The corresponding relationship of mark.
For example, can be by each tables of data be respectively set depending on the O&M engineer of networking monitoring network management scheduling system
The file that is respectively stored in of delta data.
In some embodiments, the corresponding relationship of tables of data and partition identification can be established by hash algorithm, it can be with
Respectively using the mark of each tables of data as key, the corresponding cryptographic Hash of each tables of data is obtained using hash algorithm.
It can be using the corresponding cryptographic Hash of tables of data as the corresponding partition identification of tables of data, it is thus possible to establish tables of data and subregion mark
The corresponding relationship of knowledge.
Step 502, higher level regards networking monitoring network management and dispatches system to view networking monitoring network management scheduling system, junior
System sends data read command.
In this application, data read command includes: the corresponding partition identification of each target matrix.
Step 503, junior reads out from the corresponding file of partition identification depending on networking monitoring network management scheduling system
Delta data.
In this application, for each tables of data, the delta data of tables of data is stored in corresponding corresponding to tables of data
In the file of partition identification.
In this application, receive comprising the corresponding partition identification of each target matrix data read command it
Afterwards, system is dispatched from corresponding to depending on networking monitoring network management by partition identification corresponding for each target matrix, junior
Delta data is read out in the file of partition identification.To which junior regards networking monitoring network management scheduling system respectively never
The delta data of each target matrix is read out in same file.
Step 504, higher level regards networking monitoring network management scheduling system and receives junior's view networking monitoring network management scheduling
The delta data read out that system is sent.
In this application, the change of each target matrix is read out depending on networking monitoring network management scheduling system in junior
After changing data, networking monitoring network management scheduling system can be regarded by higher level and receives junior's view networking monitoring network management scheduling
Delta data i.e. each target matrix read out by junior depending on networking monitoring network management scheduling system that system is sent
Delta data.
In some embodiments, data read command further include: the start read position for the delta data for needing to read.It rises
Beginning reading position can indicate which the last first delta data read after delta data in remaining delta data is
A byte.In other words, start read position can indicate to read and become since which byte when this reads delta data
Change data.Since the byte that start read position indicates, all changes can be read out when each reads delta data
Data.Start read position is also referred to as document location.
Illustrate to change so that higher level reads the delta data of a tables of data depending on networking monitoring network management scheduling system as an example
The process of reading data: reading order includes the corresponding partition identification of the tables of data and document location.Corresponding to the tables of data pair
There is a file under the file for the partition identification answered, all delta datas of the tables of data are stored in file.Accordingly
, document location can indicate that the last first delta data read after delta data in remaining delta data is this article
Which byte in part.
When reading out delta data from this document for the first time, document location 0, after, variation number has been read each time
According to later, higher level will be updated document location depending on networking monitoring network management scheduling system.For example, being opened for the first time from file
Begin, i.e., reads delta data since the byte of the first byte in this document i.e. the 0th.
Junior regard networking monitoring network management scheduling system receive comprising document location as 0 data read command it
Afterwards, it will be sent to from this document since all changes data in this document the first byte that document location 0 indicates
Grade view networking monitoring network management dispatches system, it is assumed that has the data of 10 bytes in this document at this time, junior regards networking monitoring connection
Net management scheduling system meeting superior view networking monitoring network management scheduling system sends the data of 10 bytes, and higher level regards prison of networking
Network management scheduling system is controlled after the data for having handled this 10 byte, document location is updated to 10.To which higher level regards networking
When Monitor-Networking management scheduling system needs to read the delta data of the tables of data next time, it is sent to junior's view networking monitoring connection
Document location in the data read command of net management scheduling system is 10, and document location is that 10 instruction juniors regard networking monitoring connection
Net management scheduling system continues to read delta data since the 10th byte, and junior regards networking monitoring network management and dispatches system
After receiving the reading order for being 10 comprising document location, since the 10th byte that document location 10 indicates, by file
In all delta data read out, then, be sent to higher level and regard networking monitoring network management scheduling system.
In this application, junior stores the tables of data of each tables of data depending on networking monitoring network management scheduling system respectively
In different subregions, i.e., the tables of data of each tables of data is respectively stored in by junior depending on networking monitoring network management scheduling system
In different files.To which, on the one hand, higher level regards networking monitoring network management scheduling system according to demand, from interested
Data are obtained in subregion, i.e., obtain data from the file that the delta data of interested tables of data is stored in, without
The delta data of all tables of data is obtained when obtaining delta data each time, thus, reduce higher level and regards networking monitoring connection
The transmission of net management scheduling system and junior's view networking monitoring network management scheduling unnecessary delta data of system, improves change
Change the read or write speed of data.On the other hand, the delta data of all tables of data is distributed to multiple subregions to store respectively, i.e.,
It is stored respectively in multiple files, facilitates the management of delta data, higher level regards networking monitoring network management and dispatches system
The delta data in any subregion, i.e., the text being stored in from the delta data of any data table memory can be obtained according to demand
The delta data that needs are obtained in part folder promotes the flexible of the business processing of higher level's view networking monitoring network management scheduling system
Property.
Data transmission method provided in an embodiment of the present invention illustrated below:
Junior regards networking monitoring network management scheduling system can be in advance as each of database tables of data distribution one
A partition identification, thus, each tables of data corresponds respectively to a partition identification.The corresponding partition identification of each tables of data
It is uniquely, the corresponding partition identification of each tables of data respectively corresponds a file in disk.
For example, distributing a partition identification 1,1 corresponding folder 1 of partition identification, the lower creation one of file 1 for tables of data 1
A file 1, then the delta data of tables of data 1 can be written into the file 1 under file 1.One is distributed for tables of data 2
Partition identification 2,2 corresponding folder 2 of partition identification create a file 2 under file 2, and the delta data of tables of data 2 can be with
It is written in the file 2 of file 2.A partition identification 3,3 corresponding folder 3 of partition identification, in file are distributed for tables of data 3
3 one file 3 of lower creation, the delta data of tables of data 3 can be written into the file 3 of file 3.A partition identification 4 is distributed,
4 corresponding folder 4 of partition identification, creates a file 4 under file 4, and the delta data of tables of data 4 can be written into file
In 4 file 4.A partition identification 5 is distributed, 5 corresponding folder 5 of partition identification creates a file 5, number under file 5
It can be written into the file 5 of file 5 according to the delta data of table 5.
Higher level is depending on networking monitoring network management scheduling system in the delta data for needing to read tables of data, it is only necessary to will feel emerging
The corresponding partition identification of tables of data belonging to the delta data of interest is sent to junior's view networking monitoring network management scheduling system, under
Grade after receiving partition identification, finds out the file corresponding to partition identification depending on networking monitoring network management scheduling system,
Then, file content, that is, interested delta data under the file for corresponding to partition identification is sent to higher level and regards networking prison
It controls network management and dispatches system, thus, higher level regards networking monitoring network management and dispatches the available interested tables of data of system
Delta data, without obtaining the delta datas of all tables of data when reading delta data each time, thus, reduce
Higher level regards networking monitoring network management scheduling system and junior's view networking monitoring network management dispatches the unnecessary variation number of system
According to transmission.
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to
According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented
Necessary to example.
With reference to Fig. 6, the structural schematic diagram of data transmission system provided in an embodiment of the present invention, data transmission system are shown
It include: that higher level regards networking monitoring network management scheduling system 601, junior's view networking monitoring network management scheduling system 602, higher level
It is passed depending on networking monitoring network management scheduling system 601 and junior's view networking monitoring network management scheduling system 602 by view networking
Transmission of data.Higher level is configured as depending on networking monitoring network management scheduling system 601: determine at least one target matrix, and
Corresponding relationship based on tables of data and partition identification determines that each of at least one target matrix target matrix is corresponding
Partition identification；Data read command is sent to junior's view networking monitoring network management scheduling system, data read command includes:
The corresponding partition identification of described each target matrix；Junior's view networking monitoring network management scheduling system is received to send
The delta data read out；Junior is configured as depending on networking monitoring network management scheduling system 602: for each number of targets
According to the corresponding partition identification of table, delta data is read out from the file for corresponding to the partition identification, wherein each mesh
The delta data for marking tables of data is pre-written into corresponding text depending on networking monitoring network management scheduling system by junior respectively
In part folder, the corresponding file of each target matrix respectively corresponds a partition identification.
In some embodiments, the corresponding relationship of the tables of data and partition identification is operated true by the configuration of user
It is fixed.
In some embodiments, the corresponding relationship of the tables of data and partition identification is established by hash algorithm.
In some embodiments, the data read command further include: position is read in the starting for the delta data for needing to read
It sets.
For system embodiments, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
A kind of data transmission method provided by the present invention, a kind of data transmission system are described in detail above,
Used herein a specific example illustrates the principle and implementation of the invention, and the explanation of above embodiments is only used
In facilitating the understanding of the method and its core concept of the invention；At the same time, for those skilled in the art, according to the present invention
Thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be construed as
Limitation of the present invention.
Claims (8)
1. a kind of data transmission method, which comprises
Higher level determines at least one target matrix depending on networking monitoring network management scheduling system, and is based on tables of data and subregion
The corresponding relationship of mark determines the corresponding partition identification of each of at least one target matrix target matrix；
Higher level regards networking monitoring network management scheduling system and reads to junior's view networking monitoring network management scheduling system transmission data
Order is taken, data read command includes: the corresponding partition identification of each described target matrix, wherein the higher level regards connection
Net Monitor-Networking management scheduling system and junior view networking monitoring network management scheduling system pass through view networking transport data；
Partition identification corresponding for each target matrix, junior view networking monitoring network management dispatch system from right
Delta data is read out in the file of partition identification described in Ying Yu, wherein the delta data of each target matrix is distinguished
It is pre-written into corresponding file by junior depending on networking monitoring network management scheduling system, each target matrix
Corresponding file respectively corresponds a partition identification；
Higher level regards networking monitoring network management scheduling system reception junior's view networking monitoring network management scheduling system and sends
The delta data read out.
2. according to the method described in claim 1, the corresponding relationship of the tables of data and partition identification is grasped by the configuration of user
Make and determines.
3. according to the method described in claim 1, the corresponding relationship of the tables of data and partition identification is established by hash algorithm.
4. method described in one of -3 according to claim 1, the data read command further include: the delta data for needing to read
Start read position.
5. a kind of data transmission system, comprising: higher level regards networking monitoring network management scheduling system, junior's view networking monitoring networking
Management scheduling system, the higher level regards networking monitoring network management scheduling system and the junior regards networking monitoring network management tune
Degree system passes through view networking transport data；
Higher level is configured as depending on networking monitoring network management scheduling system: determining at least one target matrix, and based on number
According to the corresponding relationship of table and partition identification, the corresponding subregion of each of at least one target matrix target matrix is determined
Mark；Data read command is sent to junior's view networking monitoring network management scheduling system, data read command includes: described every
The corresponding partition identification of one target matrix；Receive the reading that junior's view networking monitoring network management scheduling system is sent
Delta data out；
Junior is configured as depending on networking monitoring network management scheduling system: subregion mark corresponding for each target matrix
Know, reads out delta data from the file for corresponding to the partition identification, wherein the variation number of each target matrix
It is pre-written into corresponding file according to system is dispatched depending on networking monitoring network management by junior respectively, each target
The corresponding file of tables of data respectively corresponds a partition identification.
6. the corresponding relationship of system according to claim 5, the tables of data and partition identification is grasped by the configuration of user
Make and determines.
7. the corresponding relationship of system according to claim 5, the tables of data and partition identification is established by hash algorithm.
8. the system according to one of claim 5-7, the data read command further include: the delta data for needing to read
Start read position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910442006.7A CN110266768B (en) | 2019-05-24 | 2019-05-24 | Data transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910442006.7A CN110266768B (en) | 2019-05-24 | 2019-05-24 | Data transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110266768A true CN110266768A (en) | 2019-09-20 |
| CN110266768B CN110266768B (en) | 2022-02-25 |
Family
ID=67915380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910442006.7A Active CN110266768B (en) | 2019-05-24 | 2019-05-24 | Data transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110266768B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113987089A (en) * | 2021-12-28 | 2022-01-28 | 北京力控元通科技有限公司 | System cascading method, data processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090164721A1 (en) * | 2007-12-19 | 2009-06-25 | Nec Corporation | Hierarchical storage control apparatus, hierarchical storage control system, hierarchical storage control method, and program for controlling storage apparatus having hierarchical structure |
| US20110161588A1 (en) * | 2009-12-30 | 2011-06-30 | International Business Machines Corporation | Formation of an exclusive ownership coherence state in a lower level cache |
| CN103295464A (en) * | 2012-02-23 | 2013-09-11 | 北京四维图新科技股份有限公司 | Method and device for data management of electronic map and method and device for generation of electronic map |
| US20160191250A1 (en) * | 2014-12-31 | 2016-06-30 | Nexenta Systems, Inc. | Read-Modify-Write Processing of Chunks at the Storage Server Level in a Distributed Object Storage System |
| CN107391744A (en) * | 2017-08-10 | 2017-11-24 | 东软集团股份有限公司 | Data storage, read method, device and its equipment |
| CN109617956A (en) * | 2018-12-04 | 2019-04-12 | 视联动力信息技术股份有限公司 | A kind of data processing method and device |
-
2019
- 2019-05-24 CN CN201910442006.7A patent/CN110266768B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090164721A1 (en) * | 2007-12-19 | 2009-06-25 | Nec Corporation | Hierarchical storage control apparatus, hierarchical storage control system, hierarchical storage control method, and program for controlling storage apparatus having hierarchical structure |
| US20110161588A1 (en) * | 2009-12-30 | 2011-06-30 | International Business Machines Corporation | Formation of an exclusive ownership coherence state in a lower level cache |
| CN103295464A (en) * | 2012-02-23 | 2013-09-11 | 北京四维图新科技股份有限公司 | Method and device for data management of electronic map and method and device for generation of electronic map |
| US20160191250A1 (en) * | 2014-12-31 | 2016-06-30 | Nexenta Systems, Inc. | Read-Modify-Write Processing of Chunks at the Storage Server Level in a Distributed Object Storage System |
| CN107391744A (en) * | 2017-08-10 | 2017-11-24 | 东软集团股份有限公司 | Data storage, read method, device and its equipment |
| CN109617956A (en) * | 2018-12-04 | 2019-04-12 | 视联动力信息技术股份有限公司 | A kind of data processing method and device |
Non-Patent Citations (2)
| Title |
|---|
| MD ASAFUDDOULA: "An incremental ensemble classifier learning by means of a rule-based accuracy and diversity comparison", 《2017 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)》 * |
| 孙业勤: "基于数据分区的密度聚类算法应用研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113987089A (en) * | 2021-12-28 | 2022-01-28 | 北京力控元通科技有限公司 | System cascading method, data processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110266768B (en) | 2022-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108965224A (en) | A kind of method and apparatus of video on demand | |
| CN108632238A (en) | A kind of method and apparatus of permission control | |
| CN110475102A (en) | A kind of monitoring data processing method and system based on view networking | |
| CN109996086A (en) | A kind of view networking service method for inquiring status and device | |
| CN109617956A (en) | A kind of data processing method and device | |
| CN110536099A (en) | Alarm method, device and computer readable storage medium | |
| CN110225083A (en) | Data push method and device based on view networking | |
| CN110062295A (en) | A kind of file resource acquisition methods and system | |
| CN109284265A (en) | A kind of date storage method and system | |
| CN108307212A (en) | A kind of file order method and device | |
| CN109462594A (en) | A kind of data processing method and system based on view networking | |
| CN109766753A (en) | A kind of finger print information acquisition methods and device | |
| CN110062194A (en) | A kind of processing method of audio, video data, device and device and storage medium | |
| CN109151061A (en) | A kind of date storage method and device | |
| CN109299612A (en) | A kind of control method and device of hot-plug equipment | |
| CN110287248A (en) | A kind of data read-write method and system | |
| CN109922302A (en) | Conference reservation information acquisition method, system, equipment and machine readable media | |
| CN110022500A (en) | A kind of loss treating method and device | |
| CN110048911A (en) | A kind of Cascading Methods and server based on view networking | |
| CN110109823A (en) | A kind of application program repairing method and device | |
| CN110012316A (en) | A kind of processing method and processing device regarding networking service | |
| CN109698859A (en) | A kind of date storage method and device based on view networking | |
| CN109067895A (en) | A kind of method and apparatus remotely controlled | |
| CN110266768A (en) | A kind of data transmission method and system | |
| CN108574655A (en) | A kind of meeting prison broadcasting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |