CN114245067A - Data transmission control method, device, equipment and readable storage medium - Google Patents
Data transmission control method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN114245067A CN114245067A CN202010940486.2A CN202010940486A CN114245067A CN 114245067 A CN114245067 A CN 114245067A CN 202010940486 A CN202010940486 A CN 202010940486A CN 114245067 A CN114245067 A CN 114245067A
- Authority
- CN
- China
- Prior art keywords
- bandwidth
- data
- data acquisition
- acquisition device
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2385—Channel allocation; Bandwidth allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44245—Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a data transmission control method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: receiving an additional recording request sent by each data acquisition device; storing historical data to be uploaded in a local cache of each data acquisition device; acquiring the residual bandwidth of the network under the condition that the data receiving server receives real-time data uploaded by each data acquisition device; determining the additional bandwidth by using the network residual bandwidth; and allocating the supplementary recording bandwidth to the data acquisition equipment. According to the method, reasonable additional recording bandwidth can be distributed for the data acquisition equipment, and the additional recording is completed under the conditions that the real-time data uploading is not influenced and the uploading success rate is guaranteed.
Description
Technical Field
The present invention relates to the field of monitoring technologies, and in particular, to a data transmission control method, apparatus, device, and readable storage medium.
Background
Image monitoring tends to massive access gradually, image data transmission failure occurs when a network is abnormal under the background of a large number of front-end devices (such as image acquisition devices and monitoring and checking terminals), a complex network and a system, image data acquired in real time are cached in a local storage medium of the front-end devices before the network is recovered to be normal, and historical data cached in the front-end devices need to be retransmitted after the network is recovered to be normal, which is called supplementary recording.
However, after the network is restored, the image data acquired in real time also needs to be transmitted. At this time, the monitoring system may have a problem that the concurrent uploading of a large amount of real-time data and historical data causes data uploading failure and affects real-time service operation.
In summary, how to effectively solve the problems of data transmission and the like is a technical problem that needs to be solved urgently by those skilled in the art at present.
Disclosure of Invention
The invention aims to provide a data transmission control method, a data transmission control device, data transmission control equipment and a readable storage medium.
In order to solve the technical problems, the invention provides the following technical scheme:
a data transmission control method, comprising:
receiving an additional recording request sent by each data acquisition device; the local cache of each data acquisition device stores historical data to be uploaded;
acquiring the network residual bandwidth under the condition that the data receiving server receives the real-time data uploaded by each data acquisition device;
determining the additional bandwidth by using the network residual bandwidth;
and allocating the supplementary recording bandwidth to the data acquisition equipment.
Preferably, the acquiring the network residual bandwidth includes:
acquiring supportable bandwidth and actual used bandwidth of the switch corresponding to the data receiving server and each data acquisition device respectively;
determining the residual outgoing bandwidth of the switch by using the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server;
determining the remaining bandwidth of the switch by utilizing the supportable bandwidth and the actually used bandwidth of the switch corresponding to each data acquisition device;
and selecting the smaller value of the remaining input bandwidth of the switch and the remaining output bandwidth of the switch as the remaining bandwidth of the network.
Preferably, determining the remaining outgoing bandwidth of the switch by using the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server includes:
obtaining supportable performance of the data receiving server;
determining whether the supportable performance is greater than a performance threshold;
if not, calculating the difference value between the supportable outgoing bandwidth and the actually used outgoing bandwidth, and taking the difference value as the residual outgoing bandwidth of the switch;
and if so, determining the remaining bandwidth of the switch by combining the supportable outgoing bandwidth, the actually used outgoing bandwidth and the actually used performance.
Preferably, after the obtaining of the network residual bandwidth, before the determining of the network patch bandwidth by using the network residual bandwidth, the method further includes:
predicting the total logging time length of the historical data uploaded by each data acquisition device by combining the network residual bandwidth and the logging request;
judging whether the total length of the additional recording is greater than a length threshold value or not;
if so, enabling the appointed data acquisition equipment to stop uploading the real-time data to the data receiving server, and returning to execute the step of acquiring the network residual bandwidth.
Preferably, determining the complementary bandwidth by using the network residual bandwidth includes:
and taking the network residual bandwidth with the specified proportion as the supplementary bandwidth.
Preferably, allocating the complementary bandwidth to the data acquisition device comprises:
dividing each data acquisition device into a plurality of additional recording batches according to the priority of the data acquisition device;
and distributing the additional recording bandwidth to the data acquisition equipment according to the additional recording batches.
Preferably, allocating the additional recording bandwidth to the data acquisition device according to the additional recording batches includes:
and according to the sequence of the additional recording batches, sequentially splitting the additional recording bandwidth to the corresponding data acquisition equipment, and informing each data acquisition equipment of the specific time for uploading the historical data.
A data transmission control apparatus comprising:
the additional recording request receiving module is used for receiving the additional recording requests sent by the data acquisition equipment; the local cache of each data acquisition device stores historical data to be uploaded;
the network residual bandwidth calculation module is used for acquiring the network residual bandwidth under the condition that the data receiving server receives the real-time data uploaded by each data acquisition device;
the additional bandwidth determining module is used for determining the additional bandwidth by utilizing the network residual bandwidth;
and the supplementary recording bandwidth allocation module is used for allocating the supplementary recording bandwidth to the data acquisition equipment.
A data transmission control apparatus comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the data transmission control method when executing the computer program.
A readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described data transmission control method.
By applying the method provided by the embodiment of the invention, the additional recording requests sent by each data acquisition device are received; storing historical data to be uploaded in a local cache of each data acquisition device; acquiring the residual bandwidth of the network under the condition that the data receiving server receives real-time data uploaded by each data acquisition device; determining the additional bandwidth by using the network residual bandwidth; and allocating the supplementary recording bandwidth to the data acquisition equipment.
In the method, under the condition of receiving the additional recording requests sent by each data acquisition device, the data acquisition device can be determined to have the additional recording requirements. At this time, the network residual bandwidth may be obtained, and then the network supplementary bandwidth may be determined based on the network residual bandwidth. And allocating the additional recording bandwidth to the data acquisition equipment needing additional recording. Therefore, the data acquisition equipment can complete the additional recording based on the allocated additional recording bandwidth under the conditions of not influencing the uploading of real-time data and ensuring the uploading success rate.
Accordingly, embodiments of the present invention further provide a data transmission control apparatus, a device and a readable storage medium corresponding to the data transmission control method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of an implementation of a data transmission control method according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a data transmission control apparatus according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data transmission control device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data transmission control device according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For the sake of understanding, the following explains the related terms related to the embodiments of the present invention:
data access: data acquisition equipment such as a front-end vehicle card port or a human face card port acquires data such as pictures, videos and recordings, and uploads the data to an application system through a network.
The intelligent operation and maintenance server: the system is mainly used for operation and maintenance management, and can maintain all front-end IPCs (IP cameras) managed by the system in the environment and relevant network topologies, network information, system resource information and the like of various management devices.
A data receiving server: the system is used for assuming the data convergence access responsibility of the front-end IPC.
The central management server: the VM is a central management node of the platform, and the front-end IPC and various management devices are registered with the VM.
Referring to fig. 1, fig. 1 is a flowchart illustrating a data transmission control method according to an embodiment of the present invention, where the method is applicable to a central management server of a monitoring system. The method comprises the following steps:
s101, receiving the additional recording requests sent by the data acquisition equipment.
The local cache of each data acquisition device stores historical data to be uploaded, such as images, videos, and audio recordings.
The data acquisition equipment, namely the front-end equipment of the monitoring system, can acquire relevant data. Specifically, if the data is image data, the data acquisition device may be a camera, a video camera, an IPC, or the like; if the data is a video, the data acquisition device may be a video camera or other device capable of acquiring video; if the data is specifically a recording, the data acquisition device may specifically be a recording-enabled device such as a recorder.
After the network is recovered from the abnormal condition, if the historical data to be uploaded is stored in the local cache of the data acquisition equipment, a request of an additional recording report, namely an additional recording request, can be sent to the central management server.
That is, the history data that needs to be uploaded exists in the local cache of the data acquisition device that sent the entry supplement request. The historical data may be data collected by the data collection device in real time during the network anomaly.
And S102, acquiring the network residual bandwidth under the condition that the data receiving server receives the real-time data uploaded by each data acquisition device.
Real-time data of the data acquisition equipment or historical data needing to be subjected to additional recording need to be transmitted to reach the data receiving server. Therefore, in order to avoid the influence of the additional recording on the normal receiving of the real-time data by the data receiving server, it is required to make sure that the data receiving server obtains the network residual bandwidth under the condition of receiving the real-time data.
Specifically, the process of acquiring the network residual bandwidth includes:
the method comprises the steps of firstly, obtaining supportable bandwidth and actual used bandwidth of a switch corresponding to a data receiving server and each data acquisition device respectively;
secondly, determining the residual outgoing bandwidth of the switch by utilizing the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server;
thirdly, determining the residual incoming bandwidth of the switch by utilizing the supportable incoming bandwidth and the actually used incoming bandwidth of the switch corresponding to each data acquisition device;
and step four, selecting the smaller value of the remaining input bandwidth of the switch and the remaining output bandwidth of the switch as the remaining bandwidth of the network.
For convenience of description, the above four steps will be described in combination.
The switch corresponding to the data receiving server can support the bandwidth, namely the maximum bandwidth of the switch for sending the data to the data receiving server, and the actually used bandwidth of the switch corresponding to the data receiving server is the bandwidth size of the switch for sending the real-time data to the data receiving server under the current condition.
The manner of obtaining the supportable bandwidth of the switch may be determined according to performance parameters of each device in the system architecture, for example, directly taking the maximum data receiving bandwidth of the data receiving server as the supportable bandwidth of the switch. Of course, the bandwidth supported by the switch can also be obtained by analyzing the collected network traffic.
Specifically, by collecting the network traffic condition, the switch obtained by analysis can support the bandwidth: the method mainly comprises the steps of collecting network flow conditions, obtaining supportable outgoing bandwidth of the switch by collecting bandwidth capacity of all switches directly connected with a data receiving server, and similarly, obtaining supportable incoming bandwidth of the switch by collecting bandwidth capacity of all switches directly connected with data collecting equipment. The intelligent operation and maintenance server maintains the switch topological graph and each information, so that the number of the switches and the number of the network ports of each switch in the working state can be obtained from the intelligent operation and maintenance server. Taking an exchanger as an example, assuming that the number of network ports used by IPC in the number of used network ports is a, the number of network ports used by the data receiving server is b, and the network ports are gigabit network ports, and it is known that the network port utilization rate in the security industry is c%, then the supportable bandwidth of the exchanger can be calculated as:
supportable incoming bandwidth: a 1000 × c% (Mb/s);
supportable outgoing bandwidth: b 1000 × c% (Mb/s);
after the bandwidth which can be supported by each switch is obtained, the bandwidth which can be supported by the switches can be obtained in an accumulation mode based on the network topology structure of the monitoring system. For each switch, the smaller of the supportable ingress bandwidth or egress bandwidth can be selected to enter into the accumulation, and finally the supportable bandwidth of the switch is obtained.
Based on the supportable bandwidth and the actually used bandwidth of the switch corresponding to the data receiving server, the remaining bandwidth can be calculated. The remaining bandwidth is the bandwidth size remaining for sending data to the data receiving service relative to the switch located between the data collecting device and the data receiving server.
The supportable bandwidth and the actually used bandwidth of the switch corresponding to each data acquisition device can be acquired through historical data. It should be noted that, in the embodiment of the present invention, the supportable ingress bandwidth of the switch corresponding to each data acquisition device refers to a maximum ingress bandwidth when the switch can receive data from the data acquisition device; correspondingly, the actually used bandwidth of the switch corresponding to each data acquisition device refers to the bandwidth occupied by the switch when actually receiving the data from each data acquisition device. And subtracting the actually used incoming bandwidth from the supportable incoming bandwidth of the switch corresponding to each data acquisition device to obtain the residual incoming bandwidth. That is, the remaining incoming bandwidth refers to the bandwidth size that can also be used for receiving data sent by each data acquisition device, relative to the switch located between the data acquisition device and the data receiving server.
Because the capacity of the switch for accessing the data acquisition equipment and the capacity of the switch for accessing the data receiving server are simultaneously satisfied, the smaller value of the remaining input bandwidth of the switch and the remaining output bandwidth of the switch is selected to be used for supplementing the network remaining bandwidth of data distribution.
In one embodiment of the present application, the data receiving server takes into account not only bandwidth limitations, but also performance impacts. Even if the bandwidth meets the requirement, if the performance consumption is too large, the data receiving server can not operate normally and stably, and the service is affected. Therefore, the bandwidth and the system resource capacity are considered comprehensively, and the more reasonable bandwidth is determined and supplemented. The specific determination process of the remaining bandwidth considering the performance impact may specifically include:
step 2.1, obtaining supportable performance of the data receiving server;
step 2.2, judging whether the supportable performance is greater than a performance threshold value;
step 2.3, if not, calculating the difference value between the supportable outgoing bandwidth and the actually used outgoing bandwidth, and taking the difference value as the residual outgoing bandwidth of the switch;
and 2.4, if so, determining the residual outgoing bandwidth of the switch by combining the supportable outgoing bandwidth, the actually used outgoing bandwidth and the actually used performance.
For convenience of description, the above four steps will be described in combination.
The intelligent operation and maintenance server can collect the system resource usage (such as CPU, MEM and the like) of the data receiving server. Therefore, supportable performance of the data receiving server can be determined by the intelligent operation and maintenance server. Specifically, considering that both the bandwidth and the performance of the image receiving server are satisfied, the remaining bandwidth is calculated in two cases. Obtaining the actual used bandwidth of the data receiving server and the use condition of system resources (CPU, MEM and the like), and calculating the use size of the system resources under the condition of supportable bandwidth according to the proportion, wherein the calculation formula comprises the following steps: supportable performance-actual usage performance-supportable bandwidth/actual usage bandwidth.
The supportable performance calculated assuming that the performance is also based on not more than 90% (a preferred performance threshold, although other values may be used, such as 80%, etc.) is not more than 90%, as in the first case below, and more than 90%, as in the second case below.
The first method comprises the following steps: the bandwidth and the performance can be satisfied under the supportable bandwidth.
The remaining outgoing bandwidth can support the outgoing bandwidth-actually used outgoing bandwidth;
and the second method comprises the following steps: and under the supportable bandwidth, the bandwidth is satisfied, and the performance is not satisfied.
In this case, the bandwidth is reduced according to the performance requirement, and the calculation is as follows:
the remaining available bandwidth is the actual used bandwidth 90%/actual used performance-actual used bandwidth.
The remaining available bandwidth under different conditions is calculated through the formula, and the remaining available bandwidth can be regarded as the remaining bandwidth of the switch, so that the remaining bandwidth of the network is determined based on comparison between the remaining bandwidth of the switch and the remaining bandwidth of the switch.
And S103, determining the additional bandwidth by using the network residual bandwidth.
As can be seen from the above description of the steps, the network residual bandwidth, i.e., the bandwidth that can be used by the data receiving server to receive data. Thus, the complementary bandwidth may be determined based on the network remaining bandwidth. Generally, the additional bandwidth should be less than or equal to the remaining bandwidth of the network. That is, all or part of the network residual bandwidth may be directly determined as the complementary bandwidth.
In practical application, it is recommended to select part of the network residual bandwidth as the supplementary bandwidth. That is, a specified proportion of the network residual bandwidth may be used as the complementary bandwidth. The remaining available bandwidth selected is not recommended to be used up completely to prevent situations where the burst picture bandwidth characteristics are large (exceeding the bandwidth limit may be problematic). The additional bandwidth may therefore be a specified proportion of the network residual bandwidth, such as: the additional bandwidth is the network residual bandwidth with a specified proportion. The specific ratio may be 90%, 85%, or other specific values set or adjusted according to actual conditions.
And S104, allocating the supplementary recording bandwidth to data acquisition equipment.
After the additional bandwidth is determined, the additional bandwidth can be allocated to the data acquisition device.
Specifically, when the additional recording bandwidth is allocated to the data acquisition equipment, the additional recording bandwidth can be allocated to all the data acquisition equipment needing additional recording in an average allocation mode; the additional recording bandwidths can be sequentially distributed to the data acquisition equipment according to the priority; the ratio can be obtained according to the priority or the historical data to be uploaded, and the additional recording bandwidth can be distributed to the data acquisition equipment according to different shares.
By applying the method provided by the embodiment of the invention, the additional recording requests sent by each data acquisition device are received; storing historical data to be uploaded in a local cache of each data acquisition device; acquiring the residual bandwidth of the network under the condition that the data receiving server receives real-time data uploaded by each data acquisition device; determining the additional bandwidth by using the network residual bandwidth; and allocating the supplementary recording bandwidth to the data acquisition equipment.
In the method, under the condition of receiving the additional recording requests sent by each data acquisition device, the data acquisition device can be determined to have the additional recording requirements. At this time, the network residual bandwidth may be obtained, and then the complementary bandwidth may be determined based on the network residual bandwidth. And allocating the additional recording bandwidth to the data acquisition equipment needing additional recording. Therefore, the data acquisition equipment can complete the additional recording based on the allocated additional recording bandwidth under the conditions of not influencing the uploading of real-time data and ensuring the uploading success rate.
It should be noted that, based on the above embodiments, the embodiments of the present invention also provide corresponding improvements. In the preferred/improved embodiment, the same steps as those in the above embodiment or corresponding steps may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the preferred/improved embodiment herein.
In a specific embodiment of the present application, in consideration that in practical applications, the amount of historical data in the data acquisition device may be huge and is more important than part of the real-time data, so that when performing the logging, the requirement of uploading bandwidth of the real-time data of some data acquisition devices may be sacrificed, and the bandwidth occupied by uploading the real-time data is also used for the logging. Specifically, after acquiring the network residual bandwidth, before determining the additional bandwidth by using the network residual bandwidth, the following steps may be further performed:
the method comprises the following steps that firstly, the total supplement and record duration of historical data uploaded by each data acquisition device is estimated by combining network residual bandwidth and supplement and record requests;
judging whether the total length of the additional recording is greater than a length threshold value or not;
and step three, if so, enabling the specified data acquisition equipment to stop uploading the real-time data to the data receiving server, and returning to execute the step of acquiring the network residual bandwidth.
For convenience of description, the above three steps will be described in combination.
The designated data acquisition equipment may be data acquisition equipment corresponding to a predetermined point location that may be sacrificed. For example, a data acquisition device that is used very frequently and has a very low picture traffic can be used as the specified data acquisition device. That is, if the time for entry-adding is long and the user wants to reduce the time for entry-adding, the real-time data at the sacrificial point may stop accessing first and be cached locally, which may increase the bandwidth for entry-adding. Namely, when the residual capacity is insufficient, the bandwidth occupied by uploading the real-time data by the specified data acquisition equipment is given up to other data acquisition equipment for additional recording.
In the embodiment of the present invention, the entry-supplementing request may specifically carry entry-supplementing information, such as IPC information (e.g., IPC identifier, maximum bandwidth corresponding to IPC), data size, and picture size. Based on the information such as the residual bandwidth of the network and the data volume requested to be uploaded in the additional recording request, the total additional recording time required by the additional recording can be estimated. If the total length of the additional recording is less than or equal to the length threshold, determining that the real-time data transmission of the specified data acquisition equipment is not needed to be sacrificed; and if the total length of the additional recording is greater than the length threshold, determining that the real-time data transmission of the specified data acquisition equipment needs to be sacrificed, and further accelerating the additional recording speed.
After determining that the real-time data transmission of the designated data acquisition device needs to be sacrificed, the designated data acquisition device may stop uploading the real-time data to the data receiving server, and re-determine the network residual bandwidth under the current condition by returning to the step of acquiring the network residual bandwidth.
In an embodiment of the present application, in order to speed up the process of determining the remaining bandwidth of the network, in an embodiment, the bandwidth occupied by uploading real-time data may be estimated by performing statistics on the uploaded real-time data of the designated data acquisition device. Furthermore, on the basis of the original network residual bandwidth, the bandwidth occupied by transmitting the real-time data is added. Specifically, the bandwidth occupied by uploading the real-time data by the designated data acquisition device can be determined by counting historical data. The picture stream has a fluctuating characteristic, which can be referred to as a picture bandwidth characteristic. And (4) accessing image data, wherein pictures can be stored in a storage server, and data can be stored in a big data server. Therefore, the picture bandwidth characteristics can be collected through historical data information, the picture frequency and the number of pictures are obtained from big data, and the picture size is obtained from cloud storage, so that the picture bandwidth characteristics can be obtained. The specific acquisition mode is as follows:
inquiring the data volume accessed in a certain period of time (such as the current 2h) of a certain IPC in the big data, and recording as x; inquiring the size of the image accessed in the same IPC within the same time range in the storage device, and recording as y;
the result of the calculation of one IPC is:
x/(2 x 3600) - - - -data per second
y/(2 x 3600) - - -how many pictures bandwidth features per second
y/x-bandwidth characteristics of how much each piece of data occupies
In the above, given the picture bandwidth characteristics occupied by each IPC per second, the sum can obtain the picture bandwidth size occupied by the IPC side per second in real-time data access, that is, the actual use bandwidth of the IPC.
The bandwidth of each IPC can be calculated, so that the total bandwidth occupied by uploading real-time data by the specified data acquisition equipment can be known, and the bandwidth is the increment capable of providing the additional bandwidth.
In a specific embodiment of the present application, considering that the size of the logging-on bandwidth is limited, for data acquisition devices at different points, the importance degree and the reference value of the locally cached historical data are different, and therefore, during logging-on, the important historical data can be preferentially logged, so as to meet the actual application requirements. Specifically, allocating the complementary bandwidth to the data acquisition device may specifically include:
step one, dividing each data acquisition device into a plurality of additional recording batches according to the priority of the data acquisition device.
And step two, allocating the additional recording bandwidth to the data acquisition equipment according to the additional recording batches.
The second step may be to sequentially split the additional recording bandwidth to the corresponding data acquisition devices according to the sequence of the additional recording batches, and inform each data acquisition device of the specific time for uploading the locally cached historical data, and upload information such as available bandwidth.
It should be noted that the priority of the data acquisition device may be determined according to its specific point. The following illustrates how the priority of the data acquisition device is determined:
in this embodiment, the point locations corresponding to the data acquisition devices are classified into five categories (of course, they may also be classified into other categories) by applying experience:
1. important points are as follows: data acquisition equipment in relevant areas such as government agencies, schools and the like can be directly marked in the system.
The general data acquisition equipment has longitude and latitude information, and the data acquisition equipment radiating within a certain range corresponding to a key area on a map is regarded as a key equipment point location and can be marked as 1.
2. High frequency use of the site: the frequency of use by the user in the business application is high.
The high-utilization-rate data acquisition equipment can be obtained through a user operation log of the central management server, and meanwhile, the point position of the data acquisition equipment which is often used for research and judgment, control and the like can be obtained through inquiring big data and can be marked as 2.
3. Hot spot positions: the data volume and the flow rate of passing the vehicle and the people are large.
The access data volume of each data acquisition device in the same time period can be inquired by inquiring the big data, and if the access data volume exceeds a defined quantity value, the data acquisition device is regarded as a hot point position; or sequencing according to the access amount, and acquiring the previous camera points as hot point points, wherein the number of the hot point points can be marked as 3.
4. The sacrificial sites: the frequency of use is very low and the picture traffic is very low.
The data acquisition equipment point location which meets the requirement of ordering the last data acquisition equipment point location, the data acquisition equipment point location with the least operation log and service usage and the data acquisition equipment point location which is excluded from the key point location are selected and can be marked as 4.
5. Common point positions: points other than the above features may be labeled 5.
If 1 data acquisition device may hit multiple categories, there are multiple labels, and the priority determination is as follows:
tag 1+ tag 2+ tag 3> tag 1+ tag 2> tag 1+ tag 3> tag 2+ tag 3> tag 1> tag 2> tag 3> tag 5> tag 4.
It should be noted that, in practical applications, there are many ways to determine the priority of the data acquisition device, which are not described herein again, for example, the priority is directly set, or the priority is determined according to the frequency of the data acquired by the data acquisition device being called.
And starting round-robin from the point with the highest priority according to the priority, matching the overall topology condition in the last step, determining the network residual bandwidth and the available additional bandwidth obtained by calculating the residual performance of the data receiving server, and using the available additional bandwidth for the IPC reporting the additional requirement.
The parallel speed limitation of point data acquisition equipment with the same priority level needs to be considered. The additional recording bandwidth can be provided to the data acquisition equipment with the same priority, so that the additional recording can be simultaneously carried out, and the simultaneous completion can be ensured. And after the completion, the bandwidth can be ensured to be additionally recorded for the data acquisition equipment of the next priority.
For example, the following steps are carried out: the current environment has 5 cameras (a specific data acquisition device), which are a1, b1, c1, d1 and e1 respectively, the data amounts to be recorded are a2, b2, c2, d2 and e2 respectively, the sizes of pictures to be recorded are a3, b3, c3, d3 and e3 respectively, and the information is sent to a management platform (recording report) by IPC after the network is restored. The camera priority is calculated as a1 ═ b1 ═ c1> d1> e1, and the bandwidth available to provide the supplementary recording is d. Firstly, the a1, b1 and c1 with the highest priority are simultaneously subjected to the additional recording, and the bandwidth to be allocated is calculated to be a4, b4 and c4 respectively.
a4+b4+c4＝d
a3/a4＝b3/b4＝c3/c4
The above equation can be solved for how much bandwidth is to be allocated to a1, b1, c1, respectively. And the time consumption of the additional recording can be calculated due to the given additional recording bandwidth.
At this time, the management platform can send back responses to a1, b1 and c1 which send corresponding additional recording reports, the additional recording information is sent back to the IPC, the IPC can perform additional recording according to the bandwidth provided for the IPC after receiving the information and can basically complete the additional recording within the calculated additional recording time.
The first priority entry is still in progress while the calculation of the second priority, i.e. the bandwidth allocation of d1, can begin. After the calculation is completed, the first priority completion of the supplementary recording can be waited, and then a response is sent back to d 1; or directly give d1 a response to inform it to start the additional recording of the uploaded data in a specific time period by time-consuming calculation. and after the a1, the b1 and the c1 are completed, the d1 can be completed, and after the d1 is completed, the e1 can be completed until the completion of the supplement.
Corresponding to the above method embodiments, embodiments of the present invention further provide a data transmission control device, and the data transmission control device and the data transmission control method described above may be referred to in correspondence with each other.
Referring to fig. 2, the apparatus includes the following modules:
the additional recording request receiving module 101 is configured to receive additional recording requests sent by each data acquisition device; storing historical data to be uploaded in a local cache of each data acquisition device;
the network residual bandwidth calculating module 102 is configured to acquire a network residual bandwidth when the data receiving server receives real-time data uploaded by each data acquisition device;
the additional recording bandwidth determining module 103 is configured to determine an additional recording bandwidth by using the network residual bandwidth;
and the supplementary recording bandwidth allocation module 104 is configured to allocate the supplementary recording bandwidth to the data acquisition device.
By applying the device provided by the embodiment of the invention, the additional recording requests sent by each data acquisition device are received; storing historical data to be uploaded in a local cache of each data acquisition device; acquiring the residual bandwidth of the network under the condition that the data receiving server receives real-time data uploaded by each data acquisition device; determining the additional bandwidth by using the network residual bandwidth; and allocating the supplementary recording bandwidth to the data acquisition equipment.
In the device, under the condition of receiving the additional recording requests sent by each data acquisition device, the data acquisition devices can be determined to have additional recording requirements. At this time, the network residual bandwidth of the data receiving server may be acquired, and then the complementary bandwidth may be determined based on the network residual bandwidth. And allocating the additional recording bandwidth to the data acquisition equipment needing additional recording. Therefore, the data acquisition equipment can complete the additional recording based on the allocated additional recording bandwidth under the conditions of not influencing the uploading of real-time data and ensuring the uploading success rate.
In an embodiment of the present invention, the network residual bandwidth calculating module 102 includes:
the bandwidth information acquisition unit is used for acquiring supportable bandwidth and actual used bandwidth of the switch corresponding to the data receiving server and each data acquisition device respectively;
the residual outgoing bandwidth determining unit is used for determining the residual outgoing bandwidth of the switch by utilizing the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server;
the residual bandwidth determining unit is used for determining the residual bandwidth of the switch by utilizing the supportable bandwidth and the actually used bandwidth of the switch corresponding to each data acquisition device;
and the network residual bandwidth determining unit is used for selecting the smaller value of the switch residual incoming bandwidth and the switch residual outgoing bandwidth as the network residual bandwidth.
In an embodiment of the present invention, the remaining incoming bandwidth determining unit is specifically configured to obtain supportable performance of the data receiving server; determining whether the supportable performance is greater than a performance threshold; if not, calculating the difference value between the supportable bandwidth and the actually used bandwidth, and taking the difference value as the residual bandwidth of the switch; if so, the switch remaining out-bandwidth is determined in combination with the supportable out-bandwidth, the actual used out-bandwidth and the actual used performance.
In one embodiment of the present invention, the method further comprises:
the network residual bandwidth increment determining module is used for predicting the total supplement and record duration of the historical data uploaded by each data acquisition device by combining the network residual bandwidth and the supplement and record request before determining the supplement and record bandwidth by using the network residual bandwidth after the network residual bandwidth is obtained; judging whether the total length of the additional recording is greater than a length threshold value or not; and if so, enabling the appointed data acquisition equipment to stop uploading the real-time data to the data receiving server, and returning to execute the step of acquiring the network residual bandwidth of the data receiving server.
In an embodiment of the present invention, the complementary bandwidth determining module 103 is specifically configured to use a network residual bandwidth of a specified ratio as the complementary bandwidth.
In a specific embodiment of the present invention, the additional recording bandwidth allocating module 104 is specifically configured to divide each data acquisition device into a plurality of additional recording batches according to the priority of the data acquisition device; and allocating the additional recording bandwidth to the data acquisition equipment according to the additional recording batches.
In a specific embodiment of the present invention, the additional recording bandwidth allocation module 104 is specifically configured to sequentially split the additional recording bandwidths to the corresponding data acquisition devices according to the order of the additional recording batches, and notify each data acquisition device of the specific time for uploading the historical data.
Corresponding to the above method embodiment, an embodiment of the present invention further provides a data transmission control device, and a data transmission control device described below and a data transmission control method described above may be referred to in correspondence.
Referring to fig. 3, the data transmission control apparatus includes:
a memory 332 for storing a computer program;
a processor 322 for implementing the steps of the data transmission control method of the above-described method embodiments when executing the computer program.
Specifically, referring to fig. 4, a specific structural diagram of a data transmission control device provided in this embodiment is shown, where the data transmission control device may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, where the memory 332 stores one or more computer applications 342 or data 344. Memory 332 may be, among other things, transient or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a sequence of instructions operating on a data processing device. Still further, the central processor 322 may be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the data transmission control device 301.
The data transmission control device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341.
The steps in the data transmission control method described above may be implemented by the structure of the data transmission control apparatus. The data transmission control device may be embodied as a central management server.
Corresponding to the above method embodiment, an embodiment of the present invention further provides a readable storage medium, and a readable storage medium described below and a data transmission control method described above may be referred to in correspondence with each other.
A readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the data transmission control method of the above-mentioned method embodiment.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (10)
1. A data transmission control method, comprising:
receiving an additional recording request sent by each data acquisition device; the local cache of each data acquisition device stores historical data to be uploaded;
acquiring the network residual bandwidth under the condition that the data receiving server receives the real-time data uploaded by each data acquisition device;
determining the additional bandwidth by using the network residual bandwidth;
and allocating the supplementary recording bandwidth to the data acquisition equipment.
2. The data transmission control method according to claim 1, wherein the obtaining of the network residual bandwidth comprises:
acquiring supportable bandwidth and actual used bandwidth of the switch corresponding to the data receiving server and each data acquisition device respectively;
determining the residual outgoing bandwidth of the switch by using the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server;
determining the remaining bandwidth of the switch by utilizing the supportable bandwidth and the actually used bandwidth of the switch corresponding to each data acquisition device;
and selecting the smaller value of the remaining input bandwidth of the switch and the remaining output bandwidth of the switch as the remaining bandwidth of the network.
3. The data transmission control method according to claim 2, wherein determining the remaining outgoing bandwidth of the switch by using the supportable outgoing bandwidth and the actually used outgoing bandwidth of the switch corresponding to the data receiving server comprises:
obtaining supportable performance of the data receiving server;
determining whether the supportable performance is greater than a performance threshold;
if not, calculating the difference value between the supportable outgoing bandwidth and the actually used outgoing bandwidth, and taking the difference value as the residual outgoing bandwidth of the switch;
and if so, determining the remaining bandwidth of the switch by combining the supportable outgoing bandwidth, the actually used outgoing bandwidth and the actually used performance.
4. The data transmission control method according to claim 1, wherein after the obtaining of the network residual bandwidth, before the determining of the network supplementary bandwidth by using the network residual bandwidth, further comprises:
predicting the total logging time length of the historical data uploaded by each data acquisition device by combining the network residual bandwidth and the logging request;
judging whether the total length of the additional recording is greater than a length threshold value or not;
if so, enabling the appointed data acquisition equipment to stop uploading the real-time data to the data receiving server, and returning to execute the step of acquiring the network residual bandwidth.
5. The data transmission control method of claim 1, wherein determining the complementary bandwidth by using the network residual bandwidth comprises:
and taking the network residual bandwidth with the specified proportion as the supplementary bandwidth.
6. The data transmission control method according to any one of claims 1 to 5, wherein allocating the complementary bandwidth to the data acquisition device includes:
dividing each data acquisition device into a plurality of additional recording batches according to the priority of the data acquisition device;
and distributing the additional recording bandwidth to the data acquisition equipment according to the additional recording batches.
7. The data transmission control method according to claim 6, wherein allocating the complementary bandwidth to the data acquisition device according to the complementary lot comprises:
and according to the sequence of the additional recording batches, sequentially splitting the additional recording bandwidth to the corresponding data acquisition equipment, and informing each data acquisition equipment of the specific time for uploading the historical data.
8. A data transmission control apparatus, comprising:
the additional recording request receiving module is used for receiving the additional recording requests sent by the data acquisition equipment; the local cache of each data acquisition device stores historical data to be uploaded;
the network residual bandwidth calculation module is used for acquiring the network residual bandwidth under the condition that the data receiving server receives the real-time data uploaded by each data acquisition device;
the additional bandwidth determining module is used for determining the additional bandwidth by utilizing the network residual bandwidth;
and the supplementary recording bandwidth allocation module is used for allocating the supplementary recording bandwidth to the data acquisition equipment.
9. A data transmission control apparatus characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data transmission control method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the data transmission control method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010940486.2A CN114245067A (en) | 2020-09-09 | 2020-09-09 | Data transmission control method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010940486.2A CN114245067A (en) | 2020-09-09 | 2020-09-09 | Data transmission control method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114245067A true CN114245067A (en) | 2022-03-25 |
Family
ID=80742635
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010940486.2A Pending CN114245067A (en) | 2020-09-09 | 2020-09-09 | Data transmission control method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114245067A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115242649A (en) * | 2022-07-26 | 2022-10-25 | 白犀牛智达(北京)科技有限公司 | Data returning method and system |
-
2020
- 2020-09-09 CN CN202010940486.2A patent/CN114245067A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115242649A (en) * | 2022-07-26 | 2022-10-25 | 白犀牛智达(北京)科技有限公司 | Data returning method and system |
| CN115242649B (en) * | 2022-07-26 | 2023-10-31 | 白犀牛智达(北京)科技有限公司 | Data return method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101910537B1 (en) | Service processing method, system and device | |
| CN108924221B (en) | Method and device for allocating resources | |
| EP3361703B1 (en) | Load balancing method, related device and system | |
| CN111459750A (en) | Private cloud monitoring method and device based on non-flat network, computer equipment and storage medium | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| US9197566B2 (en) | Information processing method, recording medium, and information processing apparatus | |
| CN110545258B (en) | Streaming media server resource allocation method and device and server | |
| CN110716808B (en) | Service processing method, device, computer equipment and storage medium | |
| US20220038374A1 (en) | Microburst detection and management | |
| CN112689007B (en) | Resource allocation method, device, computer equipment and storage medium | |
| CN110740293A (en) | data transmission method, system, device and computer storage medium | |
| CN113364682A (en) | Data transmission method and device, storage medium and electronic device | |
| CN110248156B (en) | Video analysis method, platform equipment, intelligent camera and intelligent monitoring system | |
| US20150109915A1 (en) | Network traffic management | |
| KR20190030589A (en) | Method and apparatus for analyzing image of unmanned aerial vehicle through offloading system | |
| CN114245067A (en) | Data transmission control method, device, equipment and readable storage medium | |
| CN113885794B (en) | Data access method and device based on multi-cloud storage, computer equipment and medium | |
| CN109614228B (en) | Comprehensive monitoring front-end system based on dynamic load balancing mode and working method | |
| CN113840330A (en) | Method for establishing connection, gateway equipment, network system and scheduling center | |
| CN114253710A (en) | Processing method of computing request, intelligent terminal, cloud server, equipment and medium | |
| CN117241333A (en) | Data transmission system based on 5g edge calculation | |
| CN102546652B (en) | System and method for server load balancing | |
| US7783784B1 (en) | Method and apparatus for adaptive selection of algorithms to load and spread traffic on an aggregation of network interface cards | |
| CA2221527A1 (en) | Communications network monitoring | |
| CN111833478A (en) | Data processing method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |