CN115174955B

CN115174955B - Digital cinema nationwide high-speed distribution system based on future network

Info

Publication number: CN115174955B
Application number: CN202210572640.4A
Authority: CN
Inventors: 楼培德; 赵思行; 艾博; 谢明芹; 陈光琳; 张晓薇; 梁峰; 谢建平
Original assignee: Individual
Current assignee: Individual
Priority date: 2022-05-25
Filing date: 2022-05-25
Publication date: 2023-11-17
Anticipated expiration: 2042-05-25
Also published as: CN115174955A

Abstract

The application discloses a digital film nationwide high-speed issuing system based on future network, comprising: the digital film storage module is used for storing the target digital film file; the distribution management control module is used for reading the target digital movie file and transmitting the target digital movie file to the transfer module; the transfer module is used for receiving the target digital movie file transmitted by the distribution management control module, transmitting the movie file to the cinema foundation node module, receiving feedback information transmitted by the foundation node module and transmitting the feedback information to the distribution management control module; and the cinema foundation node module is used for receiving and downloading the target digital movie file transmitted by the transit module or other cinema nodes. The system can carry out national networking based on future networks, can meet the requirements of stability, safety and rapidness of ultra-large file transmission, provides massive IP addresses and strong capability and equipment for covering heterogeneous networks in a cross-domain manner, and greatly reduces the information transmission cost of the whole system.

Description

Digital cinema nationwide high-speed distribution system based on future network

Technical Field

The application relates to the technical field of movie distribution, in particular to a national high-speed distribution system of digital movies based on future networks.

Background

As the technology of movie digitization has matured, the related research and implementation of movie transmission networking has also gradually been brought up to schedule. In recent years, a perfect movie distribution network is proposed to be established in China, and digital movie network distribution is realized. The design and implementation difficulties of the national digital cinema distribution system are: firstly, the volume of a film copy file is large, and the size of a digital release film package is 200GB to 400GB, even about 500 GB; secondly, according to data statistics, by the end of 9 months in 2021, the Chinese movie market shares 14235 theatres, and if a traditional C/S architecture is adopted, huge bandwidth and hardware resources are required to realize synchronous high-speed distribution from Beijing distribution centers to all levels of nodes in the whole country, so that factors such as network bandwidth, hardware cost, transmission speed, reliability and the like are comprehensively considered in system design. Even if a novel cloud network scheme is adopted, the transmission of oversized files is still difficult to get rid of the difficult situation that three elements of the current v4 internet transmission are good in use, high in speed and low in cost, and only occupy two of the three elements at the same time.

Disclosure of Invention

The embodiment of the application provides a digital cinema nationwide high-speed distribution system based on a future network. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

The embodiment of the application provides a digital cinema nationwide high-speed distribution system based on a future network, which comprises the following components:

the digital film storage module is used for storing the target digital film file;

the distribution management control module is used for reading the target digital movie file and transmitting the target digital movie file to the transfer module;

the transfer module is used for receiving the target digital film file transmitted by the distribution management control module, transmitting the target digital film file to the cinema foundation node module, receiving feedback information transmitted by the cinema foundation node module and transmitting the feedback information to the distribution management control module;

the cinema foundation node module is used for receiving the target digital film file transmitted by the transfer module, downloading the target digital film file according to the peer-to-peer transmission mode of the node to the node, and transmitting the feedback information after the downloading to the transfer module;

and networking is performed among the distribution management control module, the transit module and the cinema foundation node module by adopting a future network IPv 9.

In some embodiments of the present application, a distribution management control module includes:

the presentation layer is used for providing a front-end operation interface, selecting a transmission task, adding the transmission task, terminating the transmission task, modifying the transmission task and inquiring the task state according to the front-end operation interface;

The interface layer is used for providing an external interface and comprises interfaces for providing information interaction with a front-end interface, transmitting task information, node configuration information, transmitting task progress information, and reporting information by a transit node and a cinema node;

the control layer is used for carrying out user permission verification and node information verification;

the business layer is used for carrying out user management, task management, node management, domain name configuration management, file management and transmission instruction issuing management;

and the database access layer is used for storing node information, transmission task information and intermediate state information.

In some embodiments of the present application, a service layer includes:

a task generating unit for generating a transmission task of the target digital cinema file and adding the transmission task to a task management list;

the task recording unit is used for recording the state of a transmission task, including recording file information issued by the transmission task, transmission target node address information, downloading path information, task starting time information, task ending time information, task state information and subtask information of the transmission task;

and the task scheduling issuing unit is used for performing task scheduling according to the execution state of the current transmission task.

In some embodiments of the present application, a service layer includes:

the metadata generation unit is used for partitioning the target movie file through a preset fingerprint algorithm to obtain file partitions, and taking the file partitions as metadata of data transmission;

a metadata recording unit for recording a time stamp, a file name, a file hash, a file chunk hash list, a command name, a file size, and a file chunk size of the target digital cinema file;

a metadata encoding and decoding unit for encoding and decoding metadata of the target digital cinema file;

and the metadata distribution unit is used for distributing the metadata to the cinema foundation node.

In some embodiments of the present application, a metadata codec unit includes:

the device comprises a buffer zone module, an encoding module, a decoding module and a linear correlation detection module;

the buffer area module is used for processing the buffer area of the encoding and decoding operation and the buffer area of the encoding coefficient, the encoding module is used for carrying out random linear network encoding, the decoding module is used for carrying out random linear network decoding, and the linear correlation detection module is used for detecting whether the received encoding block is linear irrelevant.

In some embodiments of the present application, a service layer includes:

And the node management unit is used for managing each transit node and cinema foundation node and recording node information, wherein the node information comprises a node ID, nationally unified cinema numbers, address information, node IP addresses, available port numbers, cinema aliases and whether the current cinema foundation node is online or not.

In some embodiments of the application, the transfer module comprises:

the agent unit is used for forwarding the information and the file issued by the distribution management control module to the cinema foundation node module and forwarding the information of the cinema foundation node module to the distribution management control module;

and the guiding unit is used for guiding the new cinema foundation node to join.

In some embodiments of the present application, the guiding unit is configured to generate a shared key, receive a join request of a cinema base node, send the join request of the cinema base node to the distribution management control module for verification, and if the request is valid, send the shared key to the cinema base node;

and the cinema foundation node communicates with other nodes according to the acquired shared secret key to form a private network.

In some embodiments of the application, a cinema foundation node module comprises:

the network communication unit is used for realizing message inquiry, file block uploading and file block downloading operations;

The file processing unit is used for carrying out splitting processing, merging processing, checking processing and encryption and decryption processing on the transmitted files;

the data exchange unit is used for completing data exchange between the nodes;

and the data monitoring and reporting unit is used for monitoring the downloading and uploading of the file blocks and reporting the transmission state information and the heartbeat information to the distribution management control module at regular time.

In some embodiments of the present application, the data exchange unit is configured to implement data exchange between nodes according to a peer-to-peer transmission mode of the network encoded node to node.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

the digital film nationwide high-speed release system based on the future network provided by the embodiment of the application is a nationwide film distribution system, a nationwide backbone network is constructed based on the future network IPV9 from a distribution management control module to a transfer module positioned in each province and city of the country, and networking is performed based on the future network from the transfer module to each cinema foundation node in the province. The requirement of stability, safety and rapidity of large film file transmission can be met by networking based on a future network, the capability and equipment of V9IP addresses compatible with V4 in mass and strong cross-domain coverage heterogeneous networks of the future network are provided, the information transmission cost of the whole system can be greatly reduced, and the abundant bandwidth and server resources of nodes in the heterogeneous V4vpn network can be fully utilized. The method realizes the transmission characteristics of future network heterogeneous networking, such as good use, high speed, low cost and safety.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram of a national high-speed distribution system for digital cinema based on a future network according to an illustrative embodiment;

FIG. 2 is a schematic diagram of a distribution management control module, according to an example embodiment;

FIG. 3 is a schematic diagram of a cinema foundation node module according to an example embodiment;

FIG. 4 is a schematic diagram of a private network, according to an example embodiment;

FIG. 5 is a schematic diagram of a network coding module according to an exemplary embodiment;

FIG. 6 is a schematic diagram illustrating an encoding method according to an example embodiment;

FIG. 7 is a schematic diagram illustrating a decoding method according to an exemplary embodiment;

fig. 8 is a schematic diagram illustrating a linear correlation detection method according to an exemplary embodiment.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the application to enable those skilled in the art to practice them.

It should be understood that the described embodiments are merely some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of systems and methods that are consistent with aspects of the application as detailed in the accompanying claims.

The application relates to a digital film nationwide high-speed issuing system based on a future network, which is a distributed distribution and storage system and adopts a P2P architecture to distribute digital film files, wherein the architecture utilizes bandwidth resources among cinema nodes to realize stable and rapid film distribution. Meanwhile, based on IPFS (InterPlanetary File System) as a bottom layer architecture, a file distribution process under a distributed scene is studied. By setting up a private network, DHT (Distributed Hash Table ) routing and Bitswap data exchange protocols complete data exchange between nodes. In addition, the distributed content distribution is researched and optimized by adopting the random linear network coding, a network coding module is built based on IPFS, and a data exchange protocol is modified, and simulation results prove that the node load can be reduced by adopting the random linear network coding, so that the file downloading speed is increased.

In the system implementation, a distribution management control center, a secondary transfer node and a cinema foundation transmission node are designed, and the micro-service architecture can be flexibly deployed according to network conditions. In addition, future network IPV9 networking is adopted between the systems, national backbone network and intra-provincial metropolitan area network networking are completed, test results prove that the trans-provincial transmission speed of backbone network 1G can reach 925Mbps, communication cost is extremely low, reliability and stability are good, in an area network, bandwidth requirements can be reduced through a P2P architecture, loads are reduced, high-speed distribution is realized, mass IP addresses can be provided based on a 2048-bit address and 42-layer addressing model of the future network, access requirements of a large number of cinema foundation nodes are met, devices can be identified through IPv9 addresses and IPv9 digital domain names of the devices, effective management of the devices is carried out, and a feasible solution is provided for high-speed data transmission scenes in the future network.

The system for nationally publishing digital movies based on the future network according to the embodiment of the present application will be described in detail with reference to the accompanying drawings. As shown in fig. 1, the system includes: and the digital film storage module is used for storing the target digital film file.

The national movie distribution system mainly provides content distribution and network service for digital movie copy distribution, and provides functions of storing, sharing and distributing digital movie copy files.

The system adopts a two-stage P2P architecture and is composed of a movie distribution center control server, a transfer node server and a movie foundation node service. The distribution management control center is responsible for transmitting task issuing and receiving feedback information. Meanwhile, a plurality of transfer nodes are set up to provide film node transfer, and the transfer nodes are guide nodes of each sub-distribution network and can serve as proxy servers to provide proxy services for cinema foundation nodes. Each transfer node is connected with a two-level distribution network, the network is connected with a plurality of cinema foundation terminal nodes, and digital movie files are distributed among the foundation nodes through P2P.

The digital cinema storage module in the system may be a digital cinema storage server, mainly for storing a target digital cinema file, which in one embodiment is a movie file to be distributed.

The system also includes a distribution management control module for reading the target digital cinema file and transmitting the target digital cinema file to the relay module.

The distribution management control module is a control center for the transmission and distribution of the whole film. The distribution management control module is mainly divided into a front-end service and a background service, and mainly provides an externally unified entrance for film distribution management personnel, and is used for receiving related tasks and configuration information and completing the display of related transmission states. The digital cinema transmission tasks can be centrally managed and instructed according to the schedule of the movies. The main flow is as follows:

and the manager selects the name of the film file to be transmitted on the front-end interface, then selects the secondary node and the cinema node to be transmitted, and after receiving the task, the management center issues a task instruction, and each cinema node acquires a seed file from the management center and starts P2P transmission.

As shown in fig. 2, the distribution management control module includes: the system comprises a representation layer, an interface layer, a control layer, a service layer and a database access layer.

The presentation layer is configured to provide a front-end operation interface, and the manager may perform related operations according to the front-end operation interface, for example, selecting a transmission task, adding a transmission task, terminating a transmission task, modifying a transmission task, and querying a task state. The interface layer is used for providing an external interface, and comprises interfaces for providing interaction information with a front-end interface, transmission task information, node configuration information, transmission task progress information, information reported by a transit node and a cinema node and the like. And the control layer is used for carrying out user permission verification and node information verification, such as verification of identity information of management personnel, verification of joining request information of cinema foundation nodes and the like. And the service layer is used for carrying out user management, task management, node management, file management, transmission instruction issuing management and the like and is used for realizing main service logic. And the database access layer is used for persisting node information, transmitting task information, temporarily storing intermediate state information and the like.

In the embodiment of the application, the service layer realizes the main logic of the distribution management control center, and the design and the realization of the related modules of the service layer are introduced in a key way.

The business layer comprises a task management module which mainly realizes the functions of generating, managing, instruction distributing and the like of film transmission tasks. The system comprises a task generating unit, a task management list and a task management list, wherein the task generating unit is used for generating a target digital movie file transmission task and adding the transmission task to the task management list. The related manager can select the current transmissible film file task on the front-end interface of the distribution control module, the back-end service receives the generated task information, generates the corresponding transmission task in the distribution center and adds the task into the task management table, meanwhile, as a plurality of files exist in each DCP (digital film package) file in film transmission, the subtasks are divided according to the files in the transmission, and meanwhile, subtask information is generated.

The task recording unit is used for recording the state of the transmission task, including recording the file information issued by the transmission task, the address information of the transmission target node, the download path information, the task starting time information, the task ending time information, the task state information and the subtask information of the transmission task, when the task state and the subtask state change, the task state update is received through the reporting interface, the state information is modified, the front end display is carried out, and the problem finding during the abnormality is convenient.

The system also comprises a task scheduling issuing unit used for performing task scheduling according to the execution state of the current transmission task. The task distribution process judges whether to read a new task from the task queue according to the current task transmission condition at regular time, and acquires a task to be transmitted if the task is not transmitted currently. And issuing task instructions to each province, downloading seed files from the seed management server by each region, and transmitting films by using P2P.

After generating the task, the task management module needs to schedule the task according to the current task execution state, and considering that the data volume of each transmission task is large and the transmission time is long, the transmission node hopes to execute only one transmission task at a time. Therefore, a background task scheduling management process is designed, the current task completion condition is monitored, and a new transmission task is selected according to the FIFO form. After the scheduling conditions are met, the task scheduler selects corresponding tasks and triggers a corresponding task issuing module to detect whether all the current sub-nodes are online, then the task executor issues task information to be transmitted to all the sub-nodes, and the sub-nodes execute related operations.

The service layer further includes a metadata management module, so that in order to implement P2P transmission in each cinema node, the whole DCP file is divided into a plurality of file data blocks (chrung), and each file data block is a basic unit of data transmission of the cinema base node, that is, metadata.

In some embodiments of the present application, the method includes a metadata generation unit, configured to segment a target movie file by using a preset fingerprint algorithm, obtain a file segment, and use the file segment as metadata for data transmission. For example, in the process of uploading a file, SHA-256 is used for mapping the file into a data hash block with a fixed size, meanwhile, rabin fingerprint algorithm is used for blocking the file, rabin fingerprint algorithm is used for file blocking detection, the blocking detection algorithm defines each data block fingerprint based on 20-byte secure hash algorithm, when a new data block is generated, comparison is carried out with a previous fingerprint value, whether the data block is repeated or not is judged, and if the data block is repeated, the data block is deleted repeatedly is avoided.

The metadata recording unit is further included for recording a time stamp, a file name, a file hash, a file chunk hash list, a command name, a file size, and a file chunk size of the target digital cinema file.

And the metadata encoding and decoding unit is used for encoding and decoding the metadata of the target digital cinema file. In one possible implementation, content distribution may be implemented based on a network-coded P2P approach. A metadata codec unit comprising: the device comprises a buffer zone module, an encoding module, a decoding module and a linear correlation detection module; the buffer area module is used for processing the buffer area of the encoding and decoding operation and the buffer area of the encoding coefficient, the encoding module is used for carrying out random linear network encoding, the decoding module is used for carrying out random linear network decoding, and the linear correlation detection module is used for detecting whether the received encoding block is linear irrelevant.

A metadata distribution unit is also included for distributing metadata to the cinema base nodes. After the transmission task is generated, each cinema foundation node receives the information and requests the distribution management control module to acquire the metadata information.

In some embodiments of the present application, the service layer further includes a node management unit, configured to manage each of the transit node and the cinema base node, and record node information, where the node information includes information such as a node ID, a nationally unified cinema number, address information, a node IP address, an available port number, a cinema alias, and whether the current cinema base node is online. Meanwhile, simple admission verification can be provided for cinema nodes through node management.

The national high-speed release system of the digital cinema based on the future network further comprises a transfer module, wherein the transfer module is used for receiving the target digital cinema file transmitted by the distribution management control module, sending the target digital cinema file to the cinema foundation node module, receiving feedback information transmitted by the cinema foundation node module and sending the feedback information to the distribution management control module.

In particular, the secondary relay module is typically a message relay and source DCP file server in the vicinity of each cinema node. Typically, the secondary transit node is deployed in a province, city operator room. In the overall design, based on the current IPv9 backbone network and networking conditions of all regions, the transfer module comprises an agent unit to realize a reverse agent function, and all cinema nodes communicate with a distribution management control module through the reverse agent to realize load balancing. And the system is used for forwarding the information and the file issued by the distribution management control module to the cinema foundation node module and forwarding the information of the cinema foundation node module to the distribution management control module.

A guiding unit is also included for guiding the joining of the new cinema foundation node. In some embodiments of the present application, the guiding unit is configured to generate a shared key, receive a join request of a cinema base node, send the join request of the cinema base node to the distribution management control module for verification, and send the shared key to the cinema base node if the request is valid, where the cinema base node communicates with other nodes according to the acquired shared key to form a private network.

Optionally, the system further comprises an original seed node unit, wherein the transit node can be used as an original file node of a sub-area, and initial data blocking is provided for each cinema node.

The benefit of designing the transit node is that the transit node can be deployed as desired. Through deploying the transit nodes, the P2P film distribution flexible networking can be realized, and when the bandwidth of the trans-province is insufficient, the transit nodes form a provincial sub-network to carry out provincial or indoor transmission. Meanwhile, the transit node can also serve as a nearby node to provide functions similar to a CDN (Content Delivery Network ).

The embodiment of the application also comprises a cinema foundation node module which is used for receiving and downloading the target digital film file transmitted by the transfer module and sending the feedback information after the downloading to the transfer module.

Specifically, cinema foundation nodes are deployed at each cinema and are foundation transmission nodes in a distributed transmission network, and each foundation structure is implemented based on an IPFS Node, as shown in fig. 3: cinema foundation node module comprising:

the network communication unit is used for realizing message inquiry, file block uploading and file block downloading operations depending on the bottom TCP and UDP protocols. The file processing unit is used for carrying out splitting processing, merging processing, checking processing and encryption and decryption processing on the transmitted files; the data exchange unit is used for completing data exchange between the nodes; the DHT module is realized based on Kadmelia protocol, and mainly performs functions such as node information routing, resource data block routing and the like. When the resource information changes, the index service is updated in time through the DHT module, and meanwhile, the position of the file resource is quickly positioned in the transmission process so as to realize quick transmission. And the data monitoring and reporting unit is used for monitoring the downloading and uploading of the file blocks and reporting the transmission state information and the heartbeat information to the distribution management control module at regular time.

In one possible implementation, the entire process of distributing a digital cinema file is as follows: firstly, a distribution management control module reads corresponding file catalogues and files according to distribution tasks to generate corresponding metadata files, and then sends relevant control information to cinema nodes to determine whether the cinema is on-line and the state of the cinema. The distribution management control module transmits the metadata file of the digital cinema file to be distributed through the transfer node. And after receiving the metadata file, the cinema node sends feedback to the distribution center control server through the secondary transit node server through the reverse proxy. And after receiving the control command, the cinema node starts a local P2P downloading process, and after receiving the complete file, the cinema node sends a completion feedback to the distribution center.

Because of the large size of movie files, the transmission needs to consume a large bandwidth. In order to ensure the transmission speed, a networking scheme with controllable cost and ensured transmission effect needs to be designed.

In one possible implementation manner, the distribution management control module is networked to the transfer modules in each province by adopting a future network IPV9, and a national backbone network is built through a backbone router, so that the IPv9 decimal network has the advantages of safety, controllability, compatibility, multiple address numbers, low communication cost and the like. The method has great advantages in networking, safety and other aspects, and can meet national high-speed service. Otherwise, the market price of a kilomega data private line crossing provinces can reach tens to millions, and if networking in the national range is to be realized, the cost is too high.

And a user router is deployed at each cinema foundation node, future network networking is also realized between the transit node and the cinema foundation node, and massive IP addresses can be provided through the future network networking so as to meet the joining requirements of a plurality of cinema foundation nodes.

And integrating the current network condition, and meanwhile, adopting regional P2P distribution for ensuring the transmission quality. The data movie distribution DCP package is first sent to secondary nodes of each province, which are typically deployed in the central office of each province operator. The original data file is stored in the transit node, so that the problems of network delay and the like caused by the influence of the geographic position are reduced. And the secondary nodes and the foundation nodes of each cinema form a P2P network in the area to carry out file transmission. After the links between the provinces are established, the nationwide P2P networking can be performed.

In one exemplary scenario, taking each IPFS cinema base node as an example, the overall P2P distribution process of DCP files in the system is illustrated.

First, a private network is constructed. In order to ensure the security of data transmission, the node needs to be added for verification, and a private network is built by adopting a shared key authentication mode. The nodes join the network using the shared key, allowing only each base node to connect to other nodes that have the shared key. Each node can only connect with nodes in the private network and cannot respond to information of nodes outside the private network. Malicious nodes and other attacks can be effectively avoided, and meanwhile, the communication safety in the whole P2P network is ensured.

In a private IPFS network, there are three node types:

(1) Common nodes, nodes to be subjected to data distribution.

(2) And the guiding node guides other cinema node nodes to join in the P2P network.

(3) Seed node refers to the node that holds the complete data. The functions of the guide node and the seed node are realized by the transit node. The private network formed by the nodes is shown in fig. 4. The transit node is the leading node of the whole cluster, stores all data at the same time and serves as a seed node.

In unstructured distributed systems, the joining of nodes requires a bootstrap node. To establish a private network, the bootstrap node generates a shared key while providing a transmission routing table for each node to help the node join the network faster. First, each node requests to join the network to the bootstrap node before joining the network, the bootstrap server sends a verification request to the distribution management server, and the distribution management server verifies the validity of the node in the node information table. If the node is valid, when a new node needs to join the network, the connection guiding node acquires the shared key, and after the joining node is allowed to acquire the shared key, the node can communicate with the nodes sharing the key.

The process of negotiating using the shared key is as follows: the node obtains the corresponding key and stores the corresponding key in the corresponding shared key pool. Wherein each key has a unique value. And when the client communicates each time, the client sends the ID and the key value to the server, the server searches whether the key value exists after receiving the message, if the key exists, connection is established, and if the key does not exist, the connection is disconnected.

Thus, a complete new node joining and initialization process is divided into three steps. Firstly, a newly added node pre-distributes a shared key, the node is guided to join the network through a system, and an ID corresponding to the new node is returned. And then, the new node sends a node message to the neighbor node, and after the neighbor node receives the message, the new node copies a neighbor table and sends the neighbor table to the joining node, and the neighbor node information forms an original routing table. And then, continuously sending the routing information to the neighbor nodes in the routing table, and after receiving the message, updating the own routing table and simultaneously returning the response message by the neighbor nodes. After receiving the reply message, the local routing table is updated.

If the accessed node is the first peer node in the network other than the bootstrap node, then only the newly added node is included in the routing table from the bootstrap node, at which point the subsequent seek update operation is skipped because there are temporarily no other nodes other than itself and the bootstrap node to contact.

After the private network is built successfully, the DCP seed file needs to be uploaded into the private network. Each time a DCP file is added to a network, a transfer node stores the data as an object in a local repository, the hash information of the file blocks is stored according to Merkle DAG, the hash value of each data block is stored through a distributed hash table, and then the whole data root directory is monitored.

In the process of uploading the file, the SHA-256 is used for mapping the file into a data hash block with a fixed size, meanwhile, a Rabin fingerprint algorithm is used for partitioning the file, and then the data hash block is stored in a local storage library of the IPFS node.

The file chunk detection is performed using a Rabin fingerprint algorithm that defines each data chunk fingerprint based on a 20-byte secure hash algorithm hash value. When a new data block is generated, comparing with the previous fingerprint value, and judging whether the data block is repeated. If the data block is de-duplicated, data block duplication is avoided.

Each file uploaded to the IPFS maintains the data block information of the file into a Merkle DAG based on the content identifier by adopting a directed acyclic graph, and stores and verifies the metadata information of the file and the file data block. The file name is a root hash value in the DAG, and a Merkle DAG structure of a DCP file.

In addition, the node can upload the position information of each data block to the network through the distributed hash table, and provide node discovery and data sharing service for other nodes. The process of data block content distributed storage is as follows: when a node owns a data content, the node uses SHA2-256 to convert the content identifier into the form of a routing table K bucket ID. The nodes closest to the content identifier are then obtained by DHT. The CID and address are sent to these nodes for preservation. The node receives the request to save the response information. The storage of the data block storage address information in the P2P network is completed through the operation.

After the source file is uploaded to the network, other nodes may obtain file information from the node. The distribution of the IPFS-based implementation files depends on the implementation of the underlying DHT and BitSwap.

IPFS realizes a DHT routing module based on Kadmelia, and the routing module mainly completes two parts of searching other peer nodes and content addressing. The routing module mainly provides node routing and content addressing services. The routing module operates in dependence on the node identifier and the content identifier. The node identifier and the content identifier both adopt 256bit address space, which is convenient for distance measurement.

The process of generating the node identifier by the IPFS is that a private key is randomly generated after the node is initialized, and SHA-256 hash operation is carried out on the public key corresponding to the private key to obtain the corresponding node identifier. Each node maintains a routing table to store neighbor node information, the routing table is implemented in the form of k buckets, and the node IDs of the k buckets are hash bucket IDs generated by carrying out SHA-256 operation on the peerIDs. The IPFS has 256 buckets, each bucket defaulting to storing 20 active nodes. The storage "address information" of each data block is stored by the node closest to it "distance", and this distance measure is measured by exclusive or.

(1) Node routing

And the node routing function obtains the address information of the corresponding node mainly through the node identifier. In a P2P network, since the routing table of each node is only a part of the entire network. Thus, a node should be able to find and communicate with node information that is not present in its routing table. In a P2P network, searching for other peer nodes is accomplished by iteratively searching a routing table. The main process of searching is as follows:

(a) Firstly, whether a target node exists or not is searched in a routing table of the routing table, the exclusive or distance between the current node and the target node is calculated, and the corresponding k barrels are searched according to the distance value. If so, return.

(b) If not, several nodes closest to the node are found in the k-bucket of the node, and a request is sent to the nodes to find the target node. A timeout time is set for each query. If the return node is not the target node, then recursively performing (b) until either the target node information is returned or the time times out.

And if the response information is not received after the timeout time is exceeded, replacing the candidate node and sending a new routing request.

(2) Content routing

The content routing function is mainly used for helping to acquire node information with searching data resources. Since content distribution distributes content on nodes that are "closest" to it, the manner in which content is addressed is similar to the manner in which nodes are routed.

SHA256 is used to convert the CID of the content to DHT in the same form. And searching the information of a plurality of nodes closest to the DHT of the current node, and respectively searching whether the content storage address information is stored or not by the nodes.

And then exchanging data blocks, after one DCP file is issued into a P2P network, traversing Merkle DAG consisting of the file by using Bitswap, and acquiring different parts in the graph from different peers, wherein throughput and bandwidth are not optimized.

According to the bitsawap protocol structure and messages, the entire file exchange process can be divided into the following parts:

(1) Starting a request session;

when one of the cinema nodes wants to acquire a certain file, the creation of the requested file root hash bitsawap session is triggered. The node broadcasts a WANT-HAVE wantlist message to all peer nodes that HAVE established a connection via the connection manager. At this time, the wantlist list only contains the root hash value of the file.

The node that receives the WANT _ save message adds the wantlist to the bill and checks if the data in the local database is blocked. If the data block is found, a HAVE message is sent. When the requesting node receives the HAVE message, it sends a WANT-BLOCK message to obtain the contents of the data BLOCK. The file directory root hash contains a dictionary of file data block mappings, namely Merkle DAG.

(2) Searching data blocks through Merkle DAG broadcasting;

the root hash data block points to a set of content identifiers from which the WANT-HAVE message is reconstructed. In the search for sub-blocks of data, WANT-HAVE is forwarded only to the nodes involved in the session in order to save bandwidth. Also, based on the received response HAVE or IDONT-HAVE, a decision is made as to whether to send WANT-BLOCK data BLOCKs.

The data BLOCK is retrieved by the WANT-HAVE request and transmitted using the WANT-BLOCK until the complete Merkle DAG is traversed. Once a certain data block is received, the transmitting CANCEL CANCELs the downloading of the data block.

(3) Triggering a routing operation;

after the node has consecutively received several DONT_HAVE messages, this means that all connected peer nodes currently do not store the data block being sought, at which point the node with the requested data is discovered by means of the routing module. When no data block is queried in the node database. And querying nodes with request data by using the DHT module through the neighbor node table, and finally sending a wantlist request to the queried nodes to download the data.

Aiming at the problem of the bandwidth of the current centralized distribution system, the embodiment of the application provides a system overall scheme which adopts a P2P distributed distribution scheme to improve the downloading speed and reduce the backbone bandwidth. Secondly, the design requirement of the current national digital cinema distribution system is clarified, and three parts of a distribution management control center, a cinema foundation node and a transfer node are designed by adopting a micro-service architecture for convenient deployment and maintenance. The architecture utilizes the transit node to build the sub-network of the area to complete the secondary distribution, and after the construction of the national backbone network is completed, the distributed distribution in the whole country can be realized. Finally, the IPFS-based underlying technology illustrates how to build a private network based on secondary nodes and accomplish a specific implementation of file distribution in the network, which uses the data block distribution protocol Bitswap and the distributed hash table for routing, thereby accelerating file distribution.

In an alternative embodiment, in larger-scale P2P file distribution, since a node stores only a portion of data, when a node requests certain specific resource blocks, if it cannot respond to certain requests, a higher delay may result. And as nodes increase, scheduling becomes more and more difficult. In the distributed system, the above problem can be solved by adopting random linear network coding, and the original complete file can be restored by only finding the linear combination of a plurality of original data blocks at the receiving end. This has the advantage that all data blocks are equal and the node does not need to request and wait for a particular data block.

In one possible implementation, network coding is added on the basis of a distributed architecture, as shown in fig. 5, and the network coding includes a buffer module, a coding module, a decoding module and a linear correlation detectivity module. The buffer module is mainly responsible for processing the buffer of the coding and decoding operation and the buffer of the coding coefficient. The encoding and decoding module is mainly responsible for random linear network encoding and decoding. The decoding success condition needs to satisfy two conditions: the number of received encoded blocks is greater than the number of partitioned blocks, and the received data blocks satisfy linearity independence. Therefore, the system also needs to provide a linear correlation detection module.

The P2P protocol adopts an IPFS (Internet protocol File System) bottom layer structure, and the relevant bottom layer protocols Bitswap, DHT and data organization are correspondingly adjusted to realize content distribution based on network coding.

The coding module mainly completes three functions of random linear coding, decoding and linear correlation detection of coding coefficient vectors. The performance of the encoding and decoding module is an important factor for determining the system performance, and in order to reduce the computational complexity, the encoding module adopts a sparse encoding mode. The sparse coding module can reduce coding complexity and ensure linear irrelevant probability.

The data structure of the network coding module is shown in the following table:

wherein the generation of the network codec and the random encoding vector is based on finite field operation, and the selected field size is 2 ⁸ Each operation takes 8 bits as a unit. The GcodeVector stores global code vectors of each generation for linear correlation detection. Rank is used to count the Rank of the coding matrix. GeneBuf is the buffer used for each generation for encoding and decoding, and ChrunkBuf is the buffer used for generating the chrunk data blocks.

And the coding module is used for coding the data blocks of each generation by adopting a sparse random linear coding mode in the system as shown in fig. 6. The codec operation is mainly performed independently within the Generation.

And when the data is uploaded, firstly generating a code vector by using a sparse code vector generation mode, and then linearly combining according to the code vector. If full coding is used, all the data blocks can be coded. But with sparse coding, each packet is coded with the same probability. A newly received packet may result in a spreading rate encoding of the data block in the network if it is not selected at the time of encoding.

To ensure that the coding vectors are linearly independent, a slightly greater coding density is chosen. Assuming that the number of the current data blocks to be encoded is n, the number of the data blocks received by the non-source node is k, the encoding density of the source node is (logn+d)/n, and the encoding density of the non-source node is (logn+d)/k. Meanwhile, d is set to 10, and studies prove that almost the same performance is achieved using a sparse linear coding d=10 and a fully coded system.

After the random encoding matrix is generated, the data packets with non-zero encoding coefficients are selected for linear combination.

After a certain number of encoded data packets are collected by a node, linear correlation detection is performed, and if the linearity is irrelevant, the data is decoded.

As shown in fig. 7, the decoding algorithm first determines whether the number of received data blocks is greater than the number of blocks in the original generation, and if so, takes out a certain number of encoded vectors and encoded data blocks in the generation into the buffer. When the coding coefficient matrix is linearly irrelevant, the matrix inversion and the dot product operation of the coding data block can restore the original data based on the finite field by utilizing the Gaussian elimination method to calculate the matrix inversion.

Since the data distributed in the system is a linear combination of the original data, when the linearly related data is received, the data cannot be restored to the original data, and the linear correlation detection is required in the process of receiving the data.

As shown in fig. 8, the linear correlation detection method includes that when a node receives a data block, the original intra-generation coding vector is read into a buffer, the coding vector of new received data is added, the rank of a matrix is calculated, if the rank of the matrix is equal to the number of coding vectors in the current buffer, the current coding vector is not related in linearity, the coding data packet is saved, if the rank of the matrix is smaller than the number of coding vectors, the new coding vector is related in linearity to the previous data, and the data packet is discarded.

Further, data exchange is implemented, and the system relies on the IPFS infrastructure to implement data exchange. IPFS is a highly modularized open source project with rich APIs, and a simple distribution protocol suitable for network coding is realized by modifying the Bitswap protocol.

After network coding is adopted, in the distributed data exchange process, the request is not required according to the given data block name, and only the current Generation and the number of data packets required by decoding the current Generation are required. For this purpose, the original data block attributes and the message attributes of Bitswap are modified.

Wherein the attributes of the data blocks are augmented with generation_id to identify the content identifier of the original generation of each encoded data block. After the random linear network coding is adopted, only the father hash and two parameters of the required data block are needed, and the hash value of the single data block in the wantlist is not needed. version is used to choose whether to use a network coding scheme.

Compared with bittorret and IPFS, the data packet transmitted after being encoded by adopting a random linear network is a linear combination of the original data packet. Thus, distributed content distribution node selection based on random linear network coding is relatively loose. However, the transmission and reception of the relevant blocks may reduce the performance of content distribution by network coding, requiring the use of appropriate node selection policies to reduce the transmission of the relevant data blocks.

The node selection module is used for selecting a proper client from the neighbor nodes which are connected at present to connect, and the judgment basis for connection establishment or disconnection is to judge whether the neighbor nodes have the data blocks which are irrelevant to the linearity of the node. The sparse network coding is adopted and is different from the complete coding, and the sparse network coding selects the coded data blocks with a certain probability. When there is a ring in the network topology, the transmitted data chunks may be linearly related.

When the number of data blocks of a node is small, there is a high probability that there are linearly related data blocks. Thus, on alternative neighbor node maintenance, after two nodes establish a connection, the two nodes query each other for the number of blocks that each other owns. Nodes do not request fewer data chunks than nodes except at the beginning or end of a transmission. When transmission is started or is about to end, the transmission is directly requested to the neighbor node without paying attention to the number of current data blocks. Meanwhile, in order to reduce the influence of the annular structure, a dependence detection mode is adopted to reduce the data blocking of transmission dependence. When the received data blocks are linearly correlated twice in succession, the connection is disconnected.

After selecting the appropriate node, a connection is requested to be established. Taking data request of each generation as an example, after a user requests a file, according to the current metadata information, checking how many data blocks of the generation are owned by the current node, and calculating the number of still needed data blocks. The modified exchange protocol is used to send the number of data blocks and the generation identifier needed to the neighbor node.

When a node performs an encoding operation on local data before sending a data packet, if a requested data block exists in a local database, the node calls a related encoding module to re-encode the data packet.

The transmission strategy of the data block is an important factor affecting the distribution efficiency. In bittorret transmission, a rare block-first strategy is used to download data. And judging which data is downloaded preferentially at present by exchanging the downloading block bitmap information with the connecting node. Whereas in IPFS, data download is coordinated by priority by maintaining a priority queue.

After encoding with a random linear network, the data chunks are equal in each Generation. However, the Generation is scarce. Therefore, the transmission priority is set as:

(1) The request data of the node with the least data amount among the nodes is preferentially transmitted.

(2) In the transmission data packet selection, the data block with the least number of average download data blocks of the Generation in the connection node is preferentially downloaded.

In IPFS, data chunks manage data chunk downloads through a data chunk manager. Maintaining a priority downloading queue PeerTaskQueue, managing the received data block downloading task, modifying the IPFS priority rule, and executing the priority of the downloading task. By such a flow control measure, the efficiency of the data block transmission process can be improved.

Further, when the node receives the data block meeting the requirement, it is checked whether the generation_id of the data block meets the requirement of the request, and meanwhile, linear correlation detection is performed on the encoding vector of the data block. When linearity independence is satisfied, the data block is stored to a local data store and a new encoded data block hash value is added to the metadata information file.

The data blocks are linearly combined at the nodes by random linear network coding such that each coded packet is "equal". In the distributed content distribution scene, the original data can be restored only by receiving a certain number of linearly independent data blocks at the receiving end without requesting specific data block contents. Then, a content distribution structure based on random linear network coding is designed for an IPFS infrastructure, a network coding module is added, and meanwhile, the original Bitswap protocol is modified. By adopting random linear coding to distribute the content, the network load can be optimized, and the downloading speed can be improved.

With the development of digital cinema technology, digital cinema transmission networking transmission technology is also continuously developed. In order to solve the huge bandwidth requirement and huge hardware and resource expenditure faced by the traditional distribution system adopting a C/S architecture, the application proposes networking based on a future network, and a distributed distribution storage system is adopted to finish the distribution of the digital movie, and the main technical effects are as follows:

(1) A distributed distribution architecture is proposed to accomplish large file distribution. The management center, the transfer node and the basic cinema node are designed, and the three parts adopt future network networking, so that the stability, the safety and the rapidity of system data transmission are improved, and the networking cost is reduced.

(2) And constructing a private network by adopting the pre-shared secret key to finish safe data transmission. Meanwhile, a Bitswap exchange protocol and a Kademlia distributed hash table are adopted to complete data block exchange and route positioning, and high-speed stable file distribution is realized.

(3) The sparse random linear network coding is adopted to accelerate the distributed distribution process, so that the overhead of locating, searching and coordinating scheduling of the traditional distributed distribution resources is reduced. And the bitsawap protocol is modified to support a random linear network coding based distribution process. Simulation results show that compared with traditional P2P distribution, average downloading time can be greatly reduced after random linear coding is adopted, and server load is reduced.

(4) The networking design in the whole country is completed, and the networking cost in the whole country is reduced.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims

1. A future network-based nationwide high-speed distribution system of digital cinema, comprising:

the distribution management control module is used for reading the target digital film file and transmitting the target digital film file to the transfer module;

the cinema foundation node module is used for receiving the target digital film file transmitted by the transfer module, downloading the target digital film file according to a node-to-node peer transmission mode and transmitting feedback information after the downloading to the transfer module;

wherein the distribution management control module includes: the presentation layer is used for providing a front-end operation interface, selecting a transmission task, adding the transmission task, terminating the transmission task, modifying the transmission task and inquiring the task state according to the front-end operation interface; the interface layer is used for providing an external interface and comprises interfaces for providing interaction information, transmission task information, node configuration information, transmission task progress information, transfer nodes and cinema node reporting information with the front-end operation interface; the control layer is used for carrying out user permission verification and node information verification; the business layer is used for carrying out user management, task management, node management, domain name configuration management, file management and transmission instruction issuing management; the database access layer is used for storing node information, transmission task information and intermediate state information;

The transfer module comprises: the agent unit is used for forwarding the information and the file issued by the distribution management control module to the cinema foundation node module and forwarding the information of the cinema foundation node module to the distribution management control module; the guiding unit is used for guiding the addition of the new cinema foundation node;

the cinema foundation node module comprises: the network communication unit is used for realizing message inquiry, file block uploading and file block downloading operations; the file processing unit is used for carrying out splitting processing, merging processing, checking processing and encryption and decryption processing on the transmitted files; the data exchange unit is used for completing data exchange between the nodes; the data monitoring and reporting unit is used for monitoring the downloading and uploading of the file blocks and reporting transmission state information and heartbeat information to the distribution management control module at regular time;

and networking is performed among the distribution management control module, the transfer module and the cinema foundation node module through a future network, wherein the future network is IPV9.

2. The system of claim 1, wherein the service layer comprises:

a task generating unit for generating a transmission task of a target digital cinema file and adding the transmission task to a task management list;

The task recording unit is used for recording the state of the transmission task, including recording file information, transmission target node address information, downloading path information, task starting time information, task ending time information, task state information and subtask information of the transmission task issued by the transmission task;

3. The system of claim 1, wherein the service layer comprises:

the metadata generation unit is used for blocking the target digital movie file through a preset fingerprint algorithm to obtain file blocks, and taking the file blocks as metadata of data transmission;

4. A system according to claim 3, wherein the metadata codec unit comprises:

5. The system of claim 1, wherein the service layer comprises:

the node management unit is used for managing each transit node and cinema foundation node and recording node information, wherein the node information comprises node numbers, nationally unified cinema numbers, address information, node IP addresses, available port numbers, cinema aliases and whether the current cinema foundation node is online or not.

6. The system according to claim 1, wherein the guiding unit is configured to generate a shared key, receive a join request of a cinema base node, and send the join request of the cinema base node to a distribution management control module for verification, and if the request is valid, send the shared key to the cinema base node;

And the cinema foundation node communicates with the transit node and other cinema foundation nodes according to the obtained shared secret key to form a private network.

7. The system of claim 1, wherein the data exchange unit is configured to implement data exchange between nodes according to a peer-to-peer transmission mode of the network encoded node-to-node.