CN104125271B - A kind of distributed structure/architecture method being applied to visualization data-pushing platform - Google Patents
A kind of distributed structure/architecture method being applied to visualization data-pushing platform Download PDFInfo
- Publication number
- CN104125271B CN104125271B CN201410319368.4A CN201410319368A CN104125271B CN 104125271 B CN104125271 B CN 104125271B CN 201410319368 A CN201410319368 A CN 201410319368A CN 104125271 B CN104125271 B CN 104125271B
- Authority
- CN
- China
- Prior art keywords
- back end
- master control
- data
- control node
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of distributed structure/architecture methods being applied to visualization data-pushing platform, include the following steps：（1）Basis deployment；（2）Distributed deployment；（3）Using visualization group technology, data organization is carried out；（4）Master control node is established, sets up at least two back end separately under master control node；（5）The back end of a minimum load is obtained to handle；（6）After success, the mapping of back end and sprite is established, the mapping of sprite and client is established.The method of the invention handles front end request by multiple back end, can share concurrent pressure, be effectively relieved due to concurrent and caused by wait for phenomenon；It due to having automatic dilatation and redundancy ability, can be adjusted into Mobile state by way of newly-increased or close back end when system loading changes, without shutting down, user perceives less than safe devoid of risk for entire adjustment.
Description
Technical field
The present invention relates to the application field of data visualization, in particular to a kind of being applied to visualization data-pushing platform
Distributed structure/architecture method.
Background technology
In the application field of data visualization, the front end of data exhibiting mostly uses general with the rear end Push Service for providing data
Logical C/S structures carry out the transmission of data based on ICP/IP protocol, and there are many performance bottlenecks for this mode, can support
Show front end than relatively limited, and there are Single Point of Faliures, once in application process performance reach capacity can not also accomplish it is automatic
Dilatation can only carry out dilatation by the high cost approach such as upgrading hardware.Even if using the asynchronous non-obstruction as Netty
Multi-threaded I/O pattern is also because can not be coped with big concurrent and big data transmission pair by factors such as hardware, the networks of single node framework
The impact of system.
At present using it is asynchronous it is non-obstruction multithreading carry out data exchange no doubt can effectively lifting system concurrent capability, but
It is the pattern using single node, the enabling capabilities of system will be limited to the handling capacity of network, network interface card eventually, in big concurrent scene
Under, system response time declines serious, degraded performance.The hardware resource limitation of server where being limited to simultaneously, if once
Performance can not meet business scenario needs just can only be not only of high cost by the way of this Longitudinal Extension of upgrading hardware, but also
Escalation process needs system-down to be migrated, and risk is big, also can bring adverse effect to the use of user.
Invention content
It is an object of the invention under big concurrent, big data transmission scene, height is provided for visual data exhibiting
Effect is stablized, the ability of disaster tolerance and automatic dilatation, to provide a kind of distributed structure/architecture being applied to visualization data-pushing platform
Method.
The present invention is achieved through the following technical solutions：A kind of distributed structure/architecture side being applied to visualization data-pushing platform
Method includes the following steps：
（1）Carry out the basis deployment of visualization structure；
（2）Distributed deployment is carried out on the basis of basis is disposed；
（3）The visualization group technology handled using sprite as data set with the minimum organizational unit of transmission is used, it is right
It visualizes data exhibiting and carries out data organization；
（4）Master control node is established, sets up at least two back end separately under master control node；
（5）Sprite is registered, and master control node is obtained after finding out back end working condition using load-balancing algorithm
The back end of a minimum load is obtained to handle registration request；
（6）After back end succeeds in registration, the mapping of back end and sprite is established, establishes sprite and client
Mapping.
The distributed structure/architecture method is held using sprite as minimal processing unit by using multiple back end jointly
The processing to these sprites is carried on a shoulder pole, the pressure of decomposing system is carried out with this, the expansion of back end can accomplish that the linear of performance carries
It rises, only needs to increase back end when system pressure increases.
Main controlled node is by scheduling, in the scene that new back end is added, back end removes, control sprite note
Volume information shifts in multiple back end, to complete whole system load and calculate the transfer of pressure.
Further, the step（1）In visualization structure basis deployment include the following steps：
（1.1）Initiation message is route first, monitors a serve port；
（1.2）The master control serve port of display, i.e., the listening port that previous step is started are configured, this configuration exists
It is changed in display/config/components-default.xml；
（1.3）Start display broadcast nodes services after configuration is good, broadcast nodes service can route to message after starting and send out
A registration signal is sent, a message communicating pipeline is registered, in case the push of subsequent message；
（1.4）Start client player, the IP and port, connection message route service of configuration good news routing work as visitor
Family end plays more than one when the large-size screen monitors of picture, first can the routing of whereabouts message send a registration signal, register a communication tube
Road, after channel registration, client sends picture request information by this pipeline and is route to message, and message, which is route, to be received picture and asks
After asking, the display broadcast nodes of light load are forwarded the request to by certain load balancing means, when display is broadcast
It puts after node obtains picture request information, obtains data by certain data acquisition means, then active push gives message routing,
Message routing is pushed to corresponding client by the channel between client again, after client obtains data, data wash with watercolours
Dye displaying.
Further, the step（2）In distributed deployment start multiple display on the basis of basis is disposed and broadcast
Put node, display back end extending transversely.
Further, the back end safeguards the inquiry and push of a group data set by one group of clock trigger,
The data of all back end are all pushed to master control node, then are divided propelling data according to routing table by master control node
Hair；
Further, the back end upon actuation can be from trend master control Node registry, the back end of new registration
The visualization sprite number of processing is 0, and after succeeding in registration, master control node can be by the load on the most heavy back end of load point one
It is partly transferred to new node, transfer process is first to obtain one group of sprite log-on message on origin node, these sprites are saved to new
Point registration, succeeds in registration one, and origin node removes one.
Further, further, the load-balancing algorithm is minimum load algorithm.
Visualization data exhibiting is generally divided into secondary structure and carries out data organization, is combined picture, sprite respectively.Combination
Picture is made of one or more sprites, and sprite includes the data set of various components display binding, and a sprite includes one
To multiple data sets.The main body for visualizing data-pushing is exactly these data sets, and data set is also with son in processing and transmission
Picture is grouped to carry out.Therefore minimum organizational unit of the sprite as data set processing and transmission, only to be equalized point
Cloth can effectively share calculating and the transmission pressure of system in multiple nodes.It is this using sprite as the grouping side of organizational unit
We term it visualizations to be grouped for method.
In distributed frame, there are one master control nodes and multiple back end to form, and master control node is responsible for sprite note
The routing of the load balancing, data packet of volume request, the data service etc. of distributed coordination function.Wherein load-balancing algorithm is most
Small loading algorithm, master control node can obtain the back end of a minimum load whenever having new sprite to carry out registration request
Handle new registration request, after back end succeeds in registration, establish the mapping of back end and sprite, establish sprite with
The mapping of client.
Back end is responsible for the processing of panel data set, and function is consistent with the ends the Sever function of single node version, is main
Business processing end.Back end safeguards the inquiry and push of a group data set, all data sections by one group of clock trigger
The data of point are all pushed to master control node.Propelling data is distributed according to routing table by master control node again.
The visual beggar that back end can be handled upon actuation from trend master control Node registry, the back end of new registration
Frame numbers are 0, and after succeeding in registration, the load point half on the most heavy back end of load can be transferred to new section by master control node
Point, transfer process are first to obtain one group of sprite log-on message on origin node, these sprites are registered to new node, are registered
Work(one, origin node remove one.Entire transfer process does not interfere with the normal operation of system, so in system performance deficiency
Automatic dilatation can be accomplished by only needing to open new back end.
When back end can not work normally in case of communication failure or other reasons, master control node can be shorter
It is found out in period, method is to maintain heart time by heartbeat packet.Once back end be judged as it is unavailable, on the node
All loads transfer to the node of other normal works, transfer method new node registration like above, the difference is that transfer
Load can be by being shifted again after load balancing.If the son that these transfers come can be received currently without enabled node
Picture, then all sprites can be transferred to pending queue, only arrive new back end registration, then handled.So only
It is correctly distributed, even if the failures such as superfluous machine occur for machine where back end, the available of whole system will not be influenced
Property, have good disaster tolerance.
Compared with prior art, the present invention haing the following advantages and advantageous effect：
（1）Front end request is handled by multiple back end, concurrent pressure can be shared, be effectively relieved due to concurrent and make
At waiting phenomenon.
（2）Since pressure can be shared, individual data node, need not be too advanced to the of less demanding of system resource
Server, hardware cost can be reduced effectively, even if when system pressure is big, open more server to support, it is overall at
This and flexibility are also better than upgrading the scheme of more advanced hardware in all respects.Due to having automatic dilatation and redundancy ability, it is being
System load can be adjusted by way of increasing newly or closing back end into Mobile state when changing, and entire adjustment is not necessarily to shut down,
User perceives less than safe devoid of risk.
Description of the drawings
Fig. 1 is the principle of the present invention structural schematic block diagram；
Fig. 2 is sprite register flow path figure.
Specific implementation mode
The present invention is described in further detail with reference to embodiment, embodiments of the present invention are not limited thereto.
Embodiment：
Method in the present embodiment, includes the following steps：
（1）Carry out the basis deployment of visualization structure；
The basis deployment of visualization structure includes the following steps：
（1.1）Initiation message is route first, monitors a serve port；
（1.2）The master control serve port of display, i.e., the listening port that previous step is started are configured, this configuration exists
It is changed in display/config/components-default.xml；
（1.3）Start display broadcast nodes services after configuration is good, broadcast nodes service can route to message after starting and send out
A registration signal is sent, a message communicating pipeline is registered, in case the push of subsequent message；
（1.4）Start client player, the IP and port, connection message route service of configuration good news routing work as visitor
Family end plays more than one when the large-size screen monitors of picture, first can the routing of whereabouts message send a registration signal, register a communication tube
Road, after channel registration, client sends picture request information by this pipeline and is route to message, and message, which is route, to be received picture and asks
After asking, the display broadcast nodes of light load are forwarded the request to by certain load balancing means, when display is broadcast
It puts after node obtains picture request information, obtains data by certain data acquisition means, then active push gives message routing,
Message routing is pushed to corresponding client by the channel between client again, after client obtains data, data wash with watercolours
Dye displaying.
（2）Distributed deployment is carried out on the basis of basis is disposed, starts multiple display on the basis of basis is disposed
Broadcast nodes, display back end extending transversely.
（3）The visualization group technology handled using sprite as data set with the minimum organizational unit of transmission is used, it is right
It visualizes data exhibiting and carries out data organization；
（4）Master control node is established, sets up at least two back end separately under master control node, as shown in Figure 1, visualization data
Show and be generally divided into secondary structure progress data organization, is combined picture, sprite respectively.Combined picture is by one or more sub- pictures
Face forms, and sprite includes the data set of various components display binding, and a sprite includes one or more data sets.Visualization
The main body of data-pushing is exactly these data sets, and data set is also to be grouped with sprite to carry out in processing and transmission.
Therefore sprite can have with the minimum organizational unit transmitted as long as being evenly distributed in multiple nodes as data set processing
Effect shares calculating and the transmission pressure of system.It is this that using sprite as the group technology of organizational unit, we term it visualizations point
Group.
（5）Sprite is registered, and register flow path is as shown in Fig. 2, master control node maintains heart time to visit by heartbeat packet
After primary data node working condition, the back end that a minimum load is obtained using minimum load algorithm is asked to handle registration
It asks；
（6）After back end succeeds in registration, the mapping of back end and sprite is established, establishes sprite and client
Mapping.
Distributed structure/architecture method carrys out shared pair using sprite as minimal processing unit, by using multiple back end
The processing of these sprites carrys out the pressure of decomposing system with this, and the expansion of back end can accomplish the linear promotion of performance,
System pressure only needs to increase back end when increasing.
Main controlled node is by scheduling, in the scene that new back end is added, back end removes, control sprite note
Volume information shifts in multiple back end, to complete whole system load and calculate the transfer of pressure.
Back end safeguards the inquiry and push of a group data set, the number of all back end by one group of clock trigger
Propelling data is distributed according to routing table according to being all pushed to master control node, then by master control node；
The visual beggar that back end can be handled upon actuation from trend master control Node registry, the back end of new registration
Frame numbers are 0, and after succeeding in registration, the load point half on the most heavy back end of load can be transferred to new section by master control node
Point, transfer process are first to obtain one group of sprite log-on message on origin node, these sprites are registered to new node, are registered
Work(one, origin node remove one.
In distributed frame, there are one master control nodes and multiple back end to form, and master control node is responsible for sprite note
The routing of the load balancing, data packet of volume request, the data service etc. of distributed coordination function.Wherein load-balancing algorithm is most
Small loading algorithm, master control node can obtain the back end of a minimum load whenever having new sprite to carry out registration request
Handle new registration request, after back end succeeds in registration, establish the mapping of back end and sprite, establish sprite with
The mapping of client.
Back end is responsible for the processing of panel data set, and function is consistent with the ends the Sever function of single node version, is main
Business processing end.Back end safeguards the inquiry and push of a group data set, all data sections by one group of clock trigger
The data of point are all pushed to master control node.Propelling data is distributed according to routing table by master control node again.
The visual beggar that back end can be handled upon actuation from trend master control Node registry, the back end of new registration
Frame numbers are 0, and after succeeding in registration, the load point half on the most heavy back end of load can be transferred to new section by master control node
Point, transfer process are first to obtain one group of sprite log-on message on origin node, these sprites are registered to new node, are registered
Work(one, origin node remove one.Entire transfer process does not interfere with the normal operation of system, so in system performance deficiency
Automatic dilatation can be accomplished by only needing to open new back end.
When back end can not work normally in case of communication failure or other reasons, master control node can be shorter
It is found out in period, method is to maintain heart time by heartbeat packet.Once back end be judged as it is unavailable, on the node
All loads transfer to the node of other normal works, transfer method new node registration like above, the difference is that transfer
Load can be by being shifted again after load balancing.If the son that these transfers come can be received currently without enabled node
Picture, then all sprites can be transferred to pending queue, until there is new back end registration, then handled.So only
It is correctly distributed, even if the failures such as superfluous machine occur for machine where back end, the available of whole system will not be influenced
Property, have good disaster tolerance.
The above is only presently preferred embodiments of the present invention, not does limitation in any form to the present invention, it is every according to
According to the technical spirit of the present invention to any simple modification, equivalent variations made by above example, the protection of the present invention is each fallen within
Within the scope of.
Claims (4)
1. a kind of distributed structure/architecture method being applied to visualization data-pushing platform, it is characterised in that：Include the following steps：
（1）The basis deployment for carrying out visualization structure, includes the following steps：
（1.1）Start master control node first, monitors a serve port；
（1.2）The master control serve port of configuration data node, i.e., the listening port that previous step is started, this configuration is in display/
It is changed in config/components-default.xml；
（1.3）Log-on data node serve, back end service can send a registration signal, registration after starting to master control node
One message communicating pipeline, in case the push of subsequent message；
（1.4）Log-on data node configures IP and the port of master control node on back end, connects master control node serve, when
Client player plays more than one when the large-size screen monitors of picture, which can send a registration to master control node first
Signal registers a communication pipe, and after channel registration, client player passes through this pipeline and sends picture request information to master control
Node issues back end after master control node receives picture request, after back end obtains picture request information, passes through
Data acquisition means obtain data, then active push gives master control node, master control node to be pushed away again by the channel between client
Corresponding client player is given, after client obtains data, data render is shown；
（2）Distributed deployment is carried out on the basis of basis is disposed, sets up at least two back end separately under master control node, when broadcasting
It puts client to increase, pressure increases, and individual data node is unsatisfactory for that after needing multiple back end can be directly initiated, extending transversely
Back end；
（3）The visualization grouping side with the minimum organizational unit of transmission is handled using back end using sprite as data set
Method carries out data organization to visualization data exhibiting；
（4）Sprite is registered, and master control node obtains one after finding out back end working condition, using load-balancing algorithm
The back end of a minimum load handles registration request；
（5）After back end succeeds in registration, mapping and sprite and the client player of back end and sprite are established
Mapping.
2. a kind of distributed structure/architecture method being applied to visualization data-pushing platform according to claim 1, feature
It is：The back end safeguards the inquiry and push of a group data set, all back end by one group of clock trigger
Data be all pushed to master control node, then propelling data is distributed according to routing table by master control node.
3. a kind of distributed structure/architecture method being applied to visualization data-pushing platform according to claim 1, feature
It is：The visualization that the back end can be handled upon actuation from trend master control Node registry, the back end of new registration
Sprite number is 0, and after succeeding in registration, the load point half on the most heavy back end of load can be transferred to new section by master control node
Point, transfer process are first to obtain one group of sprite log-on message on origin node, these sprites are registered to new node, are registered
Work(one, origin node remove one.
4. a kind of distributed structure/architecture method being applied to visualization data-pushing platform according to claim 1, feature
It is：The master control node maintains heart time to find out back end working condition by heartbeat packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410319368.4A CN104125271B (en) | 2014-07-07 | 2014-07-07 | A kind of distributed structure/architecture method being applied to visualization data-pushing platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410319368.4A CN104125271B (en) | 2014-07-07 | 2014-07-07 | A kind of distributed structure/architecture method being applied to visualization data-pushing platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104125271A CN104125271A (en) | 2014-10-29 |
| CN104125271B true CN104125271B (en) | 2018-07-17 |
Family
ID=51770535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410319368.4A Active CN104125271B (en) | 2014-07-07 | 2014-07-07 | A kind of distributed structure/architecture method being applied to visualization data-pushing platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104125271B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105939387A (en) * | 2016-06-23 | 2016-09-14 | 中国南方电网有限责任公司 | Multilevel load balancing and server role switching method based on network, province and district uniform dispatching |
| CN111314241B (en) * | 2020-01-17 | 2024-03-29 | 新华智云科技有限公司 | Task scheduling method and scheduling system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102123179A (en) * | 2011-03-28 | 2011-07-13 | 中国人民解放军国防科学技术大学 | Load balancing method and system applied to distributed application system |
| CN102364439A (en) * | 2011-10-19 | 2012-02-29 | 广东威创视讯科技股份有限公司 | Window rapid movement method and device of distributed processing system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102572368A (en) * | 2010-12-16 | 2012-07-11 | 中兴通讯股份有限公司 | Processing method and system of distributed video and multipoint control unit |
-
2014
- 2014-07-07 CN CN201410319368.4A patent/CN104125271B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102123179A (en) * | 2011-03-28 | 2011-07-13 | 中国人民解放军国防科学技术大学 | Load balancing method and system applied to distributed application system |
| CN102364439A (en) * | 2011-10-19 | 2012-02-29 | 广东威创视讯科技股份有限公司 | Window rapid movement method and device of distributed processing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104125271A (en) | 2014-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110113441B (en) | Computer equipment, system and method for realizing load balance | |
| US20220206908A1 (en) | Techniques for replicating state information for high availability | |
| WO2018082709A1 (en) | Method for sharing load of mobile edge computing application and controller | |
| CN109548082A (en) | Business reorientation method and device | |
| US11777848B2 (en) | Scalable routing and forwarding of packets in cloud infrastructure | |
| CN104850450A (en) | Load balancing method and system facing mixed cloud application | |
| CN104283774B (en) | The implementation method and device in a kind of AC ponds | |
| US9507625B2 (en) | Apparatus and method for generating software defined network(SDN)-based virtual network according to user demand | |
| CN105391651B (en) | Virtual optical network multi-layer resource convergence method and system | |
| CN107465616B (en) | Service routing method and device based on client | |
| CN108471629A (en) | The control method of business service quality, equipment and system in transmission network | |
| CN103220165B (en) | Processing method and device for server active downtime | |
| WO2017058641A1 (en) | Data plane manipulation in a load balancer | |
| CN110445884A (en) | A kind of container IP address management method and device | |
| Buyakar et al. | Prototyping and load balancing the service based architecture of 5G core using NFV | |
| CN106210058A (en) | A kind of reverse proxy method of multi-core parallel concurrent | |
| CN103401954B (en) | The implementation method of virtual DHCP | |
| CN104125271B (en) | A kind of distributed structure/architecture method being applied to visualization data-pushing platform | |
| US10462072B2 (en) | System and method for scaling multiclouds in a hybrid cloud architecture | |
| US11876710B2 (en) | Dynamic IP routing in a cloud environment | |
| CN103501299A (en) | Firewall cluster management method and system | |
| CN103560937B (en) | Data access method and portal server | |
| US20230370421A1 (en) | Scaling ip addresses in overlay networks | |
| WO2015069378A1 (en) | Hierarchical distribution of control information in a massively scalable network server | |
| CN115915289A (en) | Method, device, network element and medium for determining strategy or allocating resources of computing task |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |