CN112243027A - System and method for solving mass media stream distribution problem based on cluster technology - Google Patents
System and method for solving mass media stream distribution problem based on cluster technology Download PDFInfo
- Publication number
- CN112243027A CN112243027A CN202011069666.4A CN202011069666A CN112243027A CN 112243027 A CN112243027 A CN 112243027A CN 202011069666 A CN202011069666 A CN 202011069666A CN 112243027 A CN112243027 A CN 112243027A
- Authority
- CN
- China
- Prior art keywords
- source
- cluster
- edge
- media stream
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 239000010410 layer Substances 0.000 description 5
- 239000002356 single layer Substances 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Abstract
The invention discloses a system and a method for solving the problem of mass media stream distribution based on a cluster technology. The system comprises: the system comprises a source station cluster, an edge cluster, a load balancer, a stream pushing end and a stream pulling end; the method comprises the following steps: the edge instance node receives a pull stream request of a pull stream end; the edge instance node queries a database, judges whether a needed source exists in the neighbor edge instance nodes, and if the needed source exists in the neighbor edge instance nodes, the edge instance node selects the most idle neighbor edge instance node from the neighbor edge instance nodes of the source to return to the source; otherwise, directly returning from the source station cluster; and pushing the media stream to the stream pulling end after the source returning is successful. The invention has the beneficial effects that: the source station cluster and the edge cluster are compressed to a single node for deployment, the secondary forwarding capacity in the edge cluster is increased, and the media stream distribution efficiency is improved.
Description
Technical Field
The invention relates to the field of mass media stream distribution, in particular to a system and a method for solving the problem of mass media stream distribution based on a clustering technology.
Background
In the era of mobile internet, the live broadcast industry is developing at a high speed. Live broadcasting is an audio and video streaming media service process for outputting content in a video form to interact with users, when the user access amount is excessively large and the flow is suddenly increased in the live broadcasting service, a server is easily overloaded, the effectiveness and stability of the live broadcasting service are easily affected in a streaming media distribution link, and the distribution capability of the server needs to be enhanced. The streaming media distribution processing process on the streaming media server is divided into the following three links:
source station (Origin): distributing resources to an Edge (Edge) by an original site for releasing audio and video contents;
edge (Edge): the node for caching the audio and video content distributes the resources to the terminal user;
returning a source: and the edge instance node requests the required content from the source station when the content is not cached, and the source station forwards the corresponding streaming media to the edge.
Currently, there are two general server distribution processing schemes.
1. Conventional solutions
The source stations and the edges are independently deployed, the edges are deployed in a multi-layer mode, the edge source returning mode is a layer-by-layer source returning mode, the lower layer edge returns to the upper layer edge, and the top layer returns to the source stations. And the multi-source stations are deployed through fixed ip, no association exists between the source stations, and the edge nodes need to return to the source according to the fixed ip (namely, static source return). The disadvantage is that the source station is not easy to expand, when the source station is added, all edge nodes need to be informed to update the configuration of the back source address, and the emergency burst flow cannot be handled.
SRS (simple real Server) scheme
The source station server cluster and the edge server cluster are respectively constructed, so that the horizontal expansion is better supported, the server scale can be adjusted at any time according to the user quantity, and the larger-scale real-time playing is supported. The SRS is dynamically sourced back, and when the edge accesses a certain source station, if there is no required resource in the source station, a redirection instruction response 302 is returned to the edge, and a domain name of the source station requesting the resource is attached. The dynamic source returning mechanism can avoid the source returning failure problem caused by the static source returning mechanism. The single-layer edge nodes in this scheme still have limited ability to handle mass distribution.
Disclosure of Invention
In view of the above technical problems, the invention establishes the association between the source stations, compresses the source station cluster and the edge cluster to a single node for deployment, and increases the secondary forwarding capability inside the edge cluster at the same time. Therefore, the technical scheme adopted by the invention is as follows: the system and the method for solving the problem of mass media stream distribution based on the cluster technology are provided, and high-reliability mass media stream distribution processing can be carried out.
A system for solving the problem of mass media stream distribution based on cluster technology comprises: the system comprises a source station cluster, an edge cluster, a load balancer, a stream pushing end and a stream pulling end;
the source station cluster consists of a plurality of source stations, and the edge cluster consists of a plurality of edge instance nodes;
the source station cluster is an original station for releasing the media stream, is responsible for receiving the media stream sent by the stream pushing end and distributes the media stream to the edge cluster;
the edge cluster is used for caching the nodes of the media stream and is responsible for distributing the media stream received by the source station cluster to the users at the stream pulling end, and secondary forwarding capacity is increased inside the edge cluster so as to improve the efficiency of media stream distribution.
Furthermore, the source station cluster and the edge cluster are deployed on a plurality of physical machines or virtual machines in a single node, and return to the source through an internal local area network;
the load balancer comprises an uplink load balancer and a downlink load balancer, wherein the uplink load balancer is used for uplink load balancing and loading a plurality of source balancing to a plurality of source stations, the downlink load balancer is used for downlink load balancing and loading a plurality of pull flow requests to a plurality of edge instance nodes, and then downlink flow balancing of the edge cluster is formed.
The media stream pushed by the stream pushing end is called a source;
and the back source, namely the edge instance node requests the content of the required media stream from the source station cluster when the media stream is not cached, and the source station cluster forwards the corresponding streaming media to the edge cluster.
A method for solving the problem of mass media stream distribution based on the cluster technology is suitable for a system for solving the problem of mass media stream distribution based on the cluster technology, and specifically comprises the following steps:
s1: the edge instance node of the edge cluster receives a pull stream request of a pull stream end;
s2: the edge instance node queries a database, judges whether a neighbor edge instance node has a needed source, if so, executes step S3, otherwise, executes step S4;
s3: the edge instance node selects the most idle neighbor edge instance node from the neighbor edge instance nodes of the source to return to the source;
s4: directly returning from the source station cluster;
s5: and pushing the media stream to the stream pulling end after the source returning is successful.
Furthermore, the source returning mode is accurate source returning, the edge queries a database before the source returning, the position of the source station where the source is located is obtained, and then the source returning is directly performed to the source station;
the invention realizes the support of mass push flow and mass pull flow or distribution by constructing the stateless source station cluster and the multi-level edge cluster, and has the following beneficial effects:
(1) compressing the source station cluster and the edge cluster to a single node for deployment, thereby greatly improving the media distribution performance of the single node;
(2) the edge cluster increases the internal secondary forwarding capability to support the distribution of massive media streams; compared with the single-layer edge instance nodes of the SRS, the back-source scheme can increase the media distribution capacity by n times (n is the number of the edge instance nodes);
(3) according to the invention, the source station cluster can realize stateless deployment, and the media stream information is stored in the database, so that accurate source return can be realized, the source return efficiency is improved, and the efficient utilization of the source station cluster example is realized.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a system structure diagram for solving a problem of mass media stream distribution based on a clustering technique in an embodiment of the present invention;
fig. 2 is a flowchart of a method for solving a problem of mass media stream distribution based on a clustering technique in an embodiment of the present invention.
Detailed Description
For a more clear understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a system structure diagram for solving a problem of mass media stream distribution based on a clustering technique in an embodiment of the present invention.
A system for solving the problem of mass media stream distribution based on cluster technology comprises: the system comprises a source station cluster, an edge cluster, a load balancer, a stream pushing end and a stream pulling end;
the source station cluster consists of a plurality of source stations, and the edge cluster consists of a plurality of edge instance nodes;
the source station cluster is an original station for releasing the media stream, is responsible for receiving the media stream sent by the stream pushing end and distributes the media stream to the edge cluster;
the edge cluster is used for caching the nodes of the media stream and is responsible for distributing the media stream received by the source station cluster to the users at the stream pulling end;
the secondary forwarding capability is added in the edge cluster, so that the distribution efficiency of the mass media streams is improved.
The source station cluster and the edge cluster are deployed on a plurality of physical machines or virtual machines in a single node and return to the source through an internal local area network;
the media stream pushed by the stream pushing end is called a source;
the back source, namely the edge instance node requests the content of the required media stream from the source station cluster when the media stream is not cached, and the source station cluster forwards the corresponding stream media to the edge cluster;
the source station cluster can be subjected to stateless deployment, and the media stream information is stored in a database;
the source returning mode is accurate source returning, the edge cluster acquires source station information of a source through querying a database, so that the source is directly returned to the active source station, and the source returning success rate reaches 100%;
the number of instances in the source station cluster and the edge cluster can be flexibly deployed according to the number of push streams and the number of pull streams;
the load balancer is a universal load balancer and comprises an uplink load balancer and a downlink load balancer, wherein the uplink load balancer is used for uplink load balancing, a plurality of source balance loads are applied to the uplink load balancer and the downlink load balancer of a plurality of source stations for downlink load balancing, a plurality of pull flow requests are loaded to a plurality of edge instance nodes, and downlink flow balancing of edge instances is formed.
Referring to fig. 2, fig. 2 is a flowchart of a method for solving a problem of mass media stream distribution based on a clustering technique in an embodiment of the present invention.
The implementation of a method for solving the problem of mass media stream distribution based on the cluster technology specifically comprises the following steps:
s1: the edge instance node of the edge cluster receives a pull stream request of a pull stream end;
s2: the edge instance node queries a database, judges whether a neighbor edge instance node has a needed source, if so, executes step S3, otherwise, executes step S4;
s3: the edge instance node selects the most idle neighbor edge instance node from the neighbor edge instance nodes of the source to return to the source;
s4: directly returning from the source station cluster;
s5: and pushing the media stream to the stream pulling end after the source returning is successful.
The source returning mode is accurate source returning, the edge queries a database before the source returning, the position of a source station where the source is located is obtained, and then the source returning is directly performed to the source station;
the invention has the following beneficial effects:
(1) compressing the source station cluster and the edge cluster to a single node for deployment, thereby greatly improving the media distribution performance of the single node;
(2) the edge cluster increases the internal secondary forwarding capability to support the distribution of massive media streams; compared with the single-layer edge instance nodes of the SRS, the back-source scheme can increase the media distribution capacity by n times (n is the number of the edge instance nodes);
(3) according to the invention, the source station cluster can realize stateless deployment, and the media stream information is stored in the database, so that accurate source return can be realized, the source return efficiency is improved, and the efficient utilization of the source station cluster example is realized.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (5)
1. A system for solving the problem of mass media stream distribution based on cluster technology is characterized by comprising the following steps: the system comprises a source station cluster, an edge cluster, a load balancer, a stream pushing end and a stream pulling end;
the source station cluster consists of a plurality of source stations, and the edge cluster consists of a plurality of edge instance nodes;
the source station cluster is an original station for releasing the media stream, is responsible for receiving the media stream sent by the stream pushing end and distributes the media stream to the edge cluster;
the edge cluster is used for caching the nodes of the media stream and is responsible for distributing the media stream received by the source station cluster to the users at the stream pulling end;
the secondary forwarding capability is added in the edge cluster to improve the efficiency of media stream distribution.
2. The system for solving the problem of mass media stream distribution based on cluster technology as claimed in claim 1, wherein the source station cluster and the edge cluster are deployed in a single node and go back to source through an internal local area network;
and the back source, namely the edge instance node requests the content of the required media stream from the source station cluster when the media stream is not cached, and the source station cluster forwards the corresponding streaming media to the edge cluster.
3. The system according to claim 1, wherein the load balancer includes an uplink load balancer and a downlink load balancer, the uplink load balancer is configured to perform uplink load balancing to load a plurality of source stations in a balanced manner, and the downlink load balancer is configured to perform downlink load balancing to load a plurality of streaming requests to a plurality of edge instance nodes, thereby forming downlink traffic balancing of the edge cluster.
4. A method for solving the problem of mass media stream distribution based on a clustering technique, which is applicable to any one of the systems of claims 1-3 for solving the problem of mass media stream distribution based on a clustering technique, and which specifically comprises the following steps:
s1: the edge instance node of the edge cluster receives a pull stream request of a pull stream end;
s2: the edge instance node queries a database, judges whether a neighbor edge instance node has a needed source, if so, executes step S3, otherwise, executes step S4;
s3: the edge instance node selects the most idle neighbor edge instance node from the neighbor edge instance nodes of the source to return to the source;
s4: directly returning from the source station cluster;
s5: and pushing the media stream to the stream pulling end after the source returning is successful.
5. The method for solving the problem of mass media stream distribution based on the cluster technology as claimed in claim 4, wherein:
the source returning mode is accurate source returning, the edge queries a database before the source returning, the position of a source station where the source is located is obtained, and then the source returning is directly performed to the source station;
the source is a media stream pushed by the stream pushing end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069666.4A CN112243027B (en) | 2020-09-30 | 2020-09-30 | System and method for solving mass media stream distribution problem based on cluster technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069666.4A CN112243027B (en) | 2020-09-30 | 2020-09-30 | System and method for solving mass media stream distribution problem based on cluster technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112243027A true CN112243027A (en) | 2021-01-19 |
| CN112243027B CN112243027B (en) | 2023-04-07 |
Family
ID=74168625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011069666.4A Active CN112243027B (en) | 2020-09-30 | 2020-09-30 | System and method for solving mass media stream distribution problem based on cluster technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112243027B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113596105A (en) * | 2021-07-06 | 2021-11-02 | 深圳市华曦达科技股份有限公司 | Content acquisition method, edge node and computer-readable storage medium |
| CN117082267A (en) * | 2023-10-13 | 2023-11-17 | 南京感动科技有限公司 | Real-time cloud uploading system and method for full-code stream video of expressway |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106888270A (en) * | 2017-03-30 | 2017-06-23 | 网宿科技股份有限公司 | Return the method and system of source routing scheduling |
| CN109246224A (en) * | 2018-09-25 | 2019-01-18 | 麒麟合盛网络技术股份有限公司 | The containerization dispositions method and device of CDN system |
| WO2019057212A1 (en) * | 2017-09-22 | 2019-03-28 | 中兴通讯股份有限公司 | Method, apparatus and device for scheduling service within cdn node, and storage medium |
| CN110493321A (en) * | 2019-07-24 | 2019-11-22 | 网宿科技股份有限公司 | System, server are dispatched in a kind of resource acquiring method and edge |
| CN111263171A (en) * | 2020-02-25 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Live streaming media data acquisition method and edge node area networking system |
-
2020
- 2020-09-30 CN CN202011069666.4A patent/CN112243027B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106888270A (en) * | 2017-03-30 | 2017-06-23 | 网宿科技股份有限公司 | Return the method and system of source routing scheduling |
| WO2019057212A1 (en) * | 2017-09-22 | 2019-03-28 | 中兴通讯股份有限公司 | Method, apparatus and device for scheduling service within cdn node, and storage medium |
| CN109542613A (en) * | 2017-09-22 | 2019-03-29 | 中兴通讯股份有限公司 | Distribution method, device and the storage medium of service dispatch in a kind of CDN node |
| CN109246224A (en) * | 2018-09-25 | 2019-01-18 | 麒麟合盛网络技术股份有限公司 | The containerization dispositions method and device of CDN system |
| CN110493321A (en) * | 2019-07-24 | 2019-11-22 | 网宿科技股份有限公司 | System, server are dispatched in a kind of resource acquiring method and edge |
| CN111263171A (en) * | 2020-02-25 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Live streaming media data acquisition method and edge node area networking system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113596105A (en) * | 2021-07-06 | 2021-11-02 | 深圳市华曦达科技股份有限公司 | Content acquisition method, edge node and computer-readable storage medium |
| CN113596105B (en) * | 2021-07-06 | 2023-12-26 | 深圳市华曦达科技股份有限公司 | Content acquisition method, edge node and computer readable storage medium |
| CN117082267A (en) * | 2023-10-13 | 2023-11-17 | 南京感动科技有限公司 | Real-time cloud uploading system and method for full-code stream video of expressway |
| CN117082267B (en) * | 2023-10-13 | 2024-01-23 | 南京感动科技有限公司 | Real-time cloud system that goes up of highway full code stream video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112243027B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106850581B (en) | Distribution backup method, system and server for interactive live broadcast streaming media data | |
| CN100484069C (en) | File data distributing method and relative device | |
| CN112243027B (en) | System and method for solving mass media stream distribution problem based on cluster technology | |
| CN101237429B (en) | Stream media living broadcasting system, method and device based on content distribution network | |
| CN1981501A (en) | Method of scheduling the distribution of content files and cache server | |
| GB2477513A (en) | Load balancing method between streaming servers based on weighting of connection and processing loads. | |
| CN110300306B (en) | Live stream load balancing method based on RTMP protocol | |
| US8812718B2 (en) | System and method of streaming data over a distributed infrastructure | |
| CN112543357B (en) | Stream media data transmission method based on DASH protocol | |
| CN101626385A (en) | Media service method and media service system | |
| CN102055718A (en) | Method, device and system for layering request content in http streaming system | |
| CN112333108A (en) | Service scheduling method and device | |
| CN114501073A (en) | Live broadcast source returning method and device | |
| CN111726646A (en) | Method, device, equipment and storage medium for pushing video stream | |
| CN105357281A (en) | Distributed content cache access control method and system for mobile access network | |
| CN101980505A (en) | 3Tnet-based video-on-demand load balancing method | |
| CN107925845B (en) | Method and apparatus for efficient content delivery | |
| CN107920042A (en) | A kind of optimized transmission method and server of the direct broadcasting room page | |
| CN101635741A (en) | Method and system thereof for inquiring recourses in distributed network | |
| CN107306279B (en) | Data processing method, content distribution network node and resident gateway | |
| KR20150096502A (en) | Efficient cache selection for content delivery networks and user equipments | |
| US20220070507A1 (en) | Method for distributing audio/video stream in audio/video stream distribution system, and dynamic parent node | |
| CN107438098B (en) | Dynamic content distribution method and system | |
| CN110582090A (en) | CDN scheduling method, scheduling system and MEC network element | |
| CN115002497A (en) | Live broadcast source returning scheduling method and system and source returning server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A system and method based on cluster technology to solve the problem of massive media stream distribution Effective date of registration: 20231226 Granted publication date: 20230407 Pledgee: Wuhan area branch of Hubei pilot free trade zone of Bank of China Ltd. Pledgor: WUHAN XINGTU XINKE ELECTRONIC Co.,Ltd. Registration number: Y2023980073771 |