CN115208872A - Edge cloud mirror image data processing method and device, computer equipment and storage medium - Google Patents

Edge cloud mirror image data processing method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN115208872A
CN115208872A CN202110384026.0A CN202110384026A CN115208872A CN 115208872 A CN115208872 A CN 115208872A CN 202110384026 A CN202110384026 A CN 202110384026A CN 115208872 A CN115208872 A CN 115208872A
Authority
CN
China
Prior art keywords
edge
file
mirror image
service file
image data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110384026.0A
Other languages
Chinese (zh)
Inventor
程庞钢
张媛
邹艳军
巫勇明
林楷淇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SF Technology Co Ltd
Original Assignee
SF Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SF Technology Co Ltd filed Critical SF Technology Co Ltd
Priority to CN202110384026.0A priority Critical patent/CN115208872A/en
Publication of CN115208872A publication Critical patent/CN115208872A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/61—Installation
    • G06F8/63—Image based installation; Cloning; Build to order
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances

Abstract

The application relates to a method and a device for processing edge cloud mirror image data, computer equipment and a storage medium. The method comprises the following steps: acquiring a service file to be transmitted and a mirror image data transmission strategy data set; sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data; determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration; and transmitting the strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a push mirror image. By adopting the method, the utilization rate of the edge cloud network line bandwidth can be improved.

Description

Edge cloud mirror image data processing method and device, computer equipment and storage medium
Technical Field
The present application relates to the field of edge cloud technologies, and in particular, to a method and an apparatus for processing edge cloud mirror image data, a computer device, and a storage medium.
Background
Based on a series of advantages of second-level delivery, efficient deployment, elastic expansion, resource scheduling and the like of container technology, the use of containers has become a future trend. The mirror image contains software data required by the container operation, and the container operation needs to pull (pull) the TAG (generally hundreds of megabits to a few grams) of the corresponding mirror image to create the container, so the mirror image is very important for the container.
The current pull mirror TAG scheme generally creates a container through a pull official or private warehouse (i.e. a command adopted: docker domain name (optional)/mirror name: mirror: TAG (optional, default), and ensures that the container operates normally. Wherein, pull official mirror image, generally realize the acceleration through the free accelerator; pull private mirroring is typically done by building a network line on its own and then pulling the mirror through that line. However, in an edge cloud application scenario, through a pull mirror image TAG of an edge network line, there is an abnormal data transmission of the edge cloud mirror image, which further results in a low bandwidth utilization rate of the edge cloud network line.
Disclosure of Invention
In view of the foregoing, it is necessary to provide an edge cloud mirror data processing method, an edge cloud mirror data processing apparatus, a computer device, and a storage medium, which can improve the utilization rate of the line bandwidth of the edge cloud network.
An edge cloud mirroring data processing method, the method comprising:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image.
In one embodiment, the determining, in accordance with the file size of the service file and the network line bandwidth of each of the mirror image data transmission policy data, a time-consuming period required for transmitting the service file to an edge warehouse at an edge end in sequence includes:
determining a first time-consuming duration required by the business file to be transmitted from a center end to an edge warehouse of the edge end through a private network line according to the file size of the business file and the private network bandwidth of the private network line strategy data;
and determining a second time-consuming duration required for the service file to be transmitted from the central end to the download server cluster through the public network line and to be transmitted to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line strategy data.
In one embodiment, the determining the target mirror data transmission policy data from the mirror data transmission policy dataset according to each of the time-consuming durations includes:
when the first time-consuming duration is less than the second time-consuming duration, determining the private network line strategy data as target mirror image data transmission strategy data;
the transmitting the strategy data based on the target mirror image data and the pushing the service file to the edge warehouse of the edge terminal by mirror image comprises the following steps:
based on the private network line strategy data, when the size of the service file is in a first numerical value area, the service file is pushed to an edge warehouse of the edge end through triggering an idle time window of the edge end at regular time.
In one embodiment, the idle time window has a corresponding idle start time point and idle end time point, and the pushing and mirroring of the service file to the edge warehouse of the edge end through the space time window of the edge end triggered at regular time includes:
acquiring a trigger time point of the timing trigger;
and when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering the idle time window of the edge end to push the service file to the edge warehouse of the edge end through a push mirror image.
In one embodiment, the method further comprises:
and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the step of triggering the idle time window of the edge end to push the service file to the edge warehouse of the edge end through a mirror image.
In one embodiment, the determining target mirrored data transmission policy data from the mirrored data transmission policy dataset according to each of the time-consuming durations includes:
when the first time consumption duration is longer than the second time consumption duration, determining that the public network line strategy data is target mirror image data transmission strategy data;
the transmitting the strategy data based on the target mirror image data and the pushing the service file to the edge warehouse of the edge terminal by mirror image comprises the following steps:
based on the public network line strategy data, when the file size of the service file is in a second numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
uploading the compressed file to the download server cluster through a public network entrance, and generating a download address of the compressed file;
sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain the service file;
and the business file is mirrored to an edge warehouse of the edge end through pushing.
In one embodiment, the method further comprises:
based on the public network line strategy data, when the file size of the service file is in a third numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
splitting the compressed file to obtain n sub-files;
uploading the n sub-files to the download server cluster through a public network entrance, and generating download addresses of the sub-files;
sending each download address to the edge end, triggering the edge end to download the subfiles through each download address, decompressing, decrypting and combining to obtain the service file;
and mirroring the service file to an edge warehouse of the edge end through pushing.
An edge cloud mirror data processing apparatus, the apparatus comprising:
the acquisition module is used for acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
the time length determining module is used for sequentially determining the time length consumed for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
a policy data determination module for determining target mirror image data transmission policy data from the mirror image data transmission policy data set according to each of the time-consuming durations;
and the data transmission module is used for transmitting the strategy data based on the target mirror image data and transmitting the service file to the edge warehouse of the edge end through a push mirror image.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image.
According to the edge cloud mirror image data processing method, the edge cloud mirror image data processing device, the computer equipment and the storage medium, the service file to be transmitted and the mirror image data transmission strategy data set are obtained; sequentially determining time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration; transmitting strategy data based on target mirror image data, and pushing a service file to an edge warehouse of an edge end through a push mirror image; the target strategy data of the edge file transmission is determined according to the network line bandwidth and the size of the service file, so that the utilization rate of the network line bandwidth is improved, and the normal transmission of the service file is determined.
Drawings
FIG. 1 is a diagram of an application environment of a method for edge cloud mirroring data processing in an embodiment;
FIG. 2 is a schematic flow diagram of a method for edge cloud mirroring data processing in an embodiment;
FIG. 3 is a flowchart illustrating a method for data processing based on private network line policy data according to an embodiment;
FIG. 4 is a flow diagram that illustrates data processing based on public network line policy data, according to an embodiment;
FIG. 5 is a schematic flow chart illustrating a method for processing edge cloud mirror data according to another embodiment;
FIG. 6 is a block diagram of an embodiment of edge cloud mirror data processing between a center terminal and an edge terminal;
FIG. 7 is a block diagram of an edge cloud image data processing apparatus in one embodiment;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
The method for processing the edge cloud mirror image data can be applied to the application environment shown in fig. 1. Where edge end 102 communicates with hub end 104 over a network. The method comprises the steps that a central terminal obtains a service file and a mirror image data transmission strategy data set to be transmitted to an edge terminal; sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data; determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration; and transmitting the strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image. The edge terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, and tablet computers, and the central terminal 104 may be a cloud server, that is, may be implemented by an independent server or a server cluster formed by multiple servers.
In an embodiment, as shown in fig. 2, an edge cloud mirroring data processing method is provided, which is described by taking the method as an example applied to the central end in fig. 1, and includes the following steps:
step 202, acquiring a service file to be transmitted and a mirror image data transmission strategy data set.
The service file with transmission is mirror image data required by operations such as normal operation and service processing of the edge cloud service, and the edge cloud service can be but is not limited to a private mirror image. The mirror image data transmission strategy data refers to strategy data used for transmitting mirror image data between a center end and an edge end, and the mirror image data transmission strategy data comprises a network line type of data transmission and a data transmission processing mode; the mirror image data between the central end and the edge end is transmitted based on network lines, and the network lines comprise private network lines, public network lines and the like; the mirror image data transmission strategy data set at least comprises private network line strategy data and public network line strategy data, wherein the private network line strategy data can be private network delay strategy data, and the public network line strategy data can be public network real-time strategy data.
Specifically, mirror image warehouses are pre-deployed at a center end and each edge end (for example, the edge warehouses are deployed at the edge ends, and the center warehouse is deployed at the center end), a basic mirror image (including environment data on which an operating system runs and is referred by a dockerfile) is pushed to the edge end point mirror image warehouse, and a back-end server of the center end acquires a service file to be transmitted according to a file download address to be constructed by the mirror image, namely, acquires mirror image data to be transmitted; and deploying a real-time public network channel and a delayed private network channel between the central terminal and the edge terminal to obtain a preset mirror image data transmission strategy data set for transmitting mirror image data, namely private network line strategy data and public network line strategy data.
And step 204, sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data.
Specifically, attribute data of a service file is obtained, and the file size of the service file in the attribute data, a first bandwidth which can be used by a private network line for data transmission and a second bandwidth which can be used by a public network line for data transmission are determined; according to the file size of the service file, the first bandwidth in the private network line strategy data and the second bandwidth in the public network line strategy data, determining a first time-consuming duration required by transmitting the service file with the private network line strategy data according to the file size of the service file and the first bandwidth in the private network line strategy data, and determining a second time-consuming duration required by transmitting the service file with the private network line strategy data according to the file size of the service file and the second bandwidth in the public network line strategy data.
Step 206, determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration.
Specifically, by comparing the time-consuming durations required by the mirror data transmission policy data sets to transmit the same service file, the target mirror data transmission policy data may be determined from the mirror data transmission policy data set according to the time-consuming durations, but not limited to. For example, the mirror image data transmission policy data set includes private network line policy data and public network line policy data, and the image data transmission policy data with the shortest time consumption is selected from a first time consumption duration required for transmitting the service file by the private network line policy data and a second time consumption duration required for transmitting the service file by the private and public network line policy data as the target mirror image data transmission policy data.
And step 208, transmitting the strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a push mirror image.
Specifically, when the private network line strategy data is the target mirror image data transmission strategy data, the business file is pushed to an edge warehouse of an edge end through a mirror image based on the private network line strategy data; based on private network line strategy data, when the size of the service file is in a first numerical value area, the service file is mirrored to an edge warehouse of the edge end by regularly triggering an idle time window of the edge end. When the public network line strategy data is target mirror image data transmission strategy data, based on the public network line strategy data, the service file is mirrored to an edge warehouse of an edge end through pushing; based on the public network line strategy data, carrying out encryption compression processing on the file according to the size of the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file; sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file; and mirroring the service file to an edge warehouse of an edge end through pushing.
In the edge cloud mirror image data processing method, time-consuming duration required for transmitting the service file to an edge warehouse of an edge end is determined according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and target mirror image data transmission strategy data is determined from the mirror image data transmission strategy data set according to each time-consuming duration; transmitting strategy data based on target mirror image data, and pushing a service file to an edge warehouse of an edge end through a push mirror image; the target strategy data transmitted by the edge file is determined according to the network line bandwidth and the size of the service file, so that the phenomenon that the network line bandwidth is wasted due to the fact that the improper data transmission strategy data is selected and the service file is abnormally transmitted and needs to be retransmitted is avoided, the utilization rate of the network line bandwidth is improved, and the normal transmission of the service file is determined.
In an embodiment, as shown in fig. 3, a data processing method based on private network line policy data is provided, which is described by taking an example that the method is applied to a central end in fig. 1, and includes the following steps:
step 302, when the file size of the service file is in the first numerical value region, acquiring a trigger time point of the timing trigger.
The first numerical range is a file size range, which may be, but is not limited to, (10M, + ∞), where M is a unit of file size, mega. Each edge end in the private network line strategy data is preset with an idle time window (an idle starting time point and an idle ending time point are set in the idle time window), which can be but is not limited to the idle time window of each day; the service files are mirrored to the edge warehouses of the edge end through pushing by setting the idle time window, namely the service files are delayed and synchronized to each edge warehouse through the idle time window, so that the problems that the special line network used for transmitting the files between the edge end and the center end is completely used, the normal transmission of the core service files is interfered, and the utilization rate of the special line network resources is low are avoided. Optionally, the network speed limit bandwidth may also be set according to the file size of the service file and/or the importance of the service in the private network line policy data, so as to improve the utilization rate of the network while ensuring normal transmission of the service file.
Specifically, when the size of the service file is in a first numerical value region, a service file synchronization task is executed through timer triggering according to a pre-established timing task; namely, the back-end server in the central end notifies each edge end to push the service file to the mirror image (push mirror image).
Optionally, when the size of the service file is not in the first numerical region, triggering to construct a mirror image of the service file through a private line network, and pushing the mirror image to an edge warehouse at an edge end.
Step 304, judging whether the trigger time point is in the time interval of the idle starting time point and the idle ending time point, if so, executing step 306; otherwise, step 308 is performed.
And step 306, triggering an idle time window of the edge end to push the service file to an edge warehouse of the edge end through a push mirror image.
Specifically, when the trigger time point is in the time interval between the idle start time point and the idle end time point, the service file is mirrored to an edge warehouse of the edge end through pushing based on the idle time window of the edge end.
Optionally, when the trigger time point is within the time interval between the idle start time point and the idle end time point, at least two different mirror image TAG synchronization tasks (i.e., at least two different service files) are provided at the same edge, queuing is performed according to the sequence of the trigger time points of the tasks, and only one mirror image TAG synchronization task is executed at the same time, so as to ensure the accuracy of data synchronization and the integrity of data.
Step 308, when suspending the timing trigger until the next idle start time point, execute step 306.
Specifically, when the trigger time point is not within the time interval between the idle start time point and the idle end time point, suspending the timing trigger until the next idle start time point, and based on the idle time window of the edge end, mirroring the service file to the edge warehouse of the edge end through pushing. The triggering time point is not in the time interval between the idle starting time point and the idle ending time point, and may be, but not limited to, at least two different mirror image TAG synchronous tasks, and in the queuing process, when the current task is executed, the time point is caused by the idle ending time point.
In the data processing method based on private network line strategy data, an idle time window is preset at each edge end, and when the file size of a service file is in a first numerical value area, a trigger time point of timing trigger is obtained; setting an idle starting time point and an idle ending time point according to the triggering time point and the idle time window, and pushing the service file to an edge warehouse of the edge end through a push mirror image based on the idle time window of the edge end, namely utilizing the bandwidth when the service is idle, so that the utilization rate of the bandwidth is improved; the pressure on the bandwidth of the public network can be further reduced, and meanwhile, the security is higher, and the accuracy of data is ensured.
In an embodiment, as shown in fig. 4, a data processing method based on public network line policy data is provided, which is described by taking an example that the method is applied to a central end in fig. 1, and includes the following steps:
step 402, based on the policy data of the public network line, judging whether the file size of the service file is in the second numerical value area or the third numerical value area, and executing step 404 when the file size of the service file is in the second numerical value area; when the file size of the service file is in the third numerical region, step 408 is performed.
The second numerical region and the third numerical region are numerical regions with preset file sizes in the public network line strategy data, and the file size of the second numerical region is smaller than that of the third numerical region; for example, the second numerical range is (10M, 100M), and the third numerical range is [100M, + ∞ ].
Optionally, when the size of the service file is not in the second numerical region or the third numerical region, triggering to construct a mirror image of the service file through a private line network, and pushing the mirror image to an edge warehouse of an edge end.
And step 404, encrypting and compressing the service file to obtain a compressed file, uploading the compressed file to the download server cluster through the public network entrance, and generating a download address of the compressed file.
Specifically, when the size of the service file is in the second numerical value area, the service file is encrypted and compressed in a lossless manner to obtain a complete compressed file, and the compressed file is uploaded to a central-end public network download server cluster to obtain a safe download address URL _ ID and a file signature value MD5 of the compressed file; the method comprises the steps that a central-end public network downloading server cluster is established at a central-end public network inlet, the public network downloading server cluster comprises at least 3 public network downloading servers, the public network downloading server cluster supports breakpoint continuous transmission of files, and the size of the transmitted files is larger than or equal to 10M.
Step 406, sending the download address to the edge, triggering the edge to download the compressed file via the download address, decompressing and decrypting to obtain the service file, and executing step 414.
Specifically, the central end sends the generated download address to the edge end through a private network or a virtual private network, which may be one edge end or a plurality of edge ends; generating a trigger instruction, wherein the trigger instruction is used for triggering the edge terminal to download the compressed file through the download address and carry out decompression and decryption processing to obtain a service file; when the edge terminal receives the download address, the compressed file and the corresponding dockerfile template are downloaded through the download address according to the trigger instruction, the file signature value MD5 is verified, and when the verification is passed, the compressed file is subjected to decompression and decryption operation, and the service file to be transmitted is obtained.
And step 408, encrypting and compressing the service file to obtain a compressed file, and splitting the compressed file to obtain n sub-files.
Specifically, when the size of the service file is in the third numerical region, the service file is encrypted and compressed in a lossless manner to obtain a complete compressed file, and the compressed file is split into n sub-files with file sizes not in the third numerical region.
And step 410, uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files.
Specifically, the n sub-files are uploaded to different download servers in the download server cluster through a public network entry, and a safe download address URL _ ID and a file signature value MD5 of each sub-file are obtained.
Step 412, sending each download address to the edge, triggering the edge to download the subfile through each download address, decompressing, decrypting and combining to obtain the service file, and executing step 414.
Specifically, the central end sends the generated download addresses of the subfiles to the edge end through a private network or a virtual private network, which may be one edge end or a plurality of edge ends; generating a trigger instruction, wherein the trigger instruction is used for triggering the edge terminal to download the corresponding subfiles through the download address, and performing combination, decompression and decryption processing on the subfiles to obtain a service file; when the edge terminal receives the download address, the sub-file and the corresponding dockerfile template are downloaded through the download address according to the trigger instruction, the file signature value MD5 is verified, when the verification is passed, the sub-files are combined to obtain a compressed file, and the compressed file is subjected to decompression and decryption operation to obtain the service file to be transmitted.
And step 414, the business file is mirrored to an edge warehouse of the edge end through pushing.
Specifically, the edge end pushes the constructed mirror image to an edge warehouse of the edge end by constructing the mirror image of the service file, generates a message of successful pushing and feeds the message back to the center end; and when the central terminal receives the message fed back by the edge terminal, generating a deleting instruction, and deleting the files of the public network download server cluster according to the deleting instruction.
According to the data processing method based on the public network line strategy data, by building a download server of a public network, when the file size of a service file is in a second numerical value area, encryption compression processing is carried out on the service file to obtain a compressed file, and when the file size of the service file is in a third numerical value area, encryption compression processing is carried out on the service file to obtain the compressed file, so that the safety of a mirror image file in the Internet and the utilization rate of network bandwidth are improved; the compressed file or the n sub-files are uploaded to a download server set through a public network entrance for distribution, so that huge cost caused by the fact that the file is too large due to special line expansion is avoided; the compressed file is split to obtain n sub-files, so that the downloading pressure of a single downloading server is reduced, and the bandwidth utilization rate is improved.
In another embodiment, as shown in fig. 5, an edge cloud mirroring data processing method is provided, which is described by taking the method as an example applied to the central end in fig. 1, and includes the following steps:
step 502, acquiring a service file to be transmitted and a mirror image data transmission strategy data set.
Step 504, determining a first time-consuming duration required for transmitting the service file from the center end to the edge warehouse of the edge end through the private network line according to the file size of the service file and the private network bandwidth of the private network line policy data.
Step 506, according to the file size of the service file and the public network bandwidth of the public network line policy data, determining a second time-consuming duration required for the service file to be transmitted from the center end to the download server cluster through the public network line and to be transmitted to the edge warehouse of the edge end through the download server.
Step 508, determine whether the first time-consuming duration is less than the second time-consuming duration, if yes, go to step 510, otherwise, go to step 514.
Step 510, determining that the private network line policy data is the target mirror image data transmission policy data.
And step 512, based on the private network line strategy data, the service file is mirrored to an edge warehouse of the edge end through pushing.
Specifically, when the file size of the service file is in a first numerical value area, the service file is pushed to an edge warehouse of an edge end through a pushing mirror image by regularly triggering an idle time window of the edge end; namely acquiring a trigger time point of timing trigger; when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to push the business file to an edge warehouse of the edge end through a push mirror image; and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the idle time window of the triggering edge end again to push the service file to an edge warehouse of the edge end through a push mirror image.
And step 514, determining the public network line strategy data as target mirror image data transmission strategy data.
And 516, based on the public network line policy data, mirroring the service file to an edge warehouse of the edge terminal by pushing.
Specifically, based on the public network line strategy data, when the file size of the service file is in a second numerical value area, the service file is encrypted and compressed to obtain a compressed file; uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file; sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file; the business file is mirrored to an edge warehouse of an edge end through pushing; based on the public network line strategy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file; splitting the compressed file to obtain n sub-files; uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files; sending each download address to an edge terminal, triggering the edge terminal to download the subfiles through each download address, decompressing, decrypting and combining to obtain a service file; and mirroring the service file to an edge warehouse of an edge end through pushing.
The following is a block diagram of a structure of edge cloud image data processing between the center end and the edge end in one embodiment, as shown in fig. 6. The method comprises the steps that a download server of a public network is set up on the side of a central end, a rear-end server and a central warehouse are arranged, the download server, the rear-end server and the central warehouse are in network connection through an intranet (local area network), a two-way private network (2-6M two-way access network) and a one-way private network (more than 10M +) are arranged between an edge end A, an edge end B and an edge end B, the edge warehouse A and the edge warehouse B in the edge end A are sequentially arranged, communication is carried out between the edge warehouse A and the edge nodes 1 and 2 in the edge end A through the intranet in the edge end, and communication is carried out between the edge warehouse B in the edge end B, the edge nodes 3 and the edge nodes 4 through the intranet in the edge end.
Acquiring a service file to be transmitted and a mirror image data transmission strategy data set, and determining a first time-consuming duration required by the service file to be transmitted from a center end to an edge warehouse of an edge end through a private network line; determining a second time-consuming duration required by the business file transmitted from the center end to the download server cluster through the public network line and transmitted to an edge warehouse of the edge end through the download server, and acquiring a trigger time point of timing trigger when the first time-consuming duration is less than the second time-consuming duration and the file size of the business file is in a first numerical region; when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing; and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the idle time window of the triggering edge end again to mirror the service file to an edge warehouse of the edge end through pushing.
When the first time consumption duration is longer than the second time consumption duration, based on the public network line strategy data, when the file size of the service file is in a second numerical value area, encrypting and compressing the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file; sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file; the business file is mirrored to an edge warehouse of an edge end through pushing; based on the public network line strategy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file; splitting the compressed file to obtain n sub-files; uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files; sending each download address to an edge terminal, triggering the edge terminal to download the subfiles through each download address, decompressing, decrypting and combining to obtain a service file; and mirroring the service file to an edge warehouse of an edge end through pushing.
According to the edge cloud mirror image data processing method, a service file to be transmitted is obtained, the time-consuming duration required by the public network line strategy data and the private network line strategy data transmission file is determined in sequence according to the size of the service file to be transmitted, the most appropriate mirror image data transmission strategy data is determined according to the time-consuming duration, when the strategy data is transmitted by the public network line strategy data, the security of the mirror image file in the internet and the utilization rate of network bandwidth are improved by carrying out encryption compression processing on the file, the file is split and uploaded to a download server set for distribution, the huge cost caused by the fact that the file is too large due to the capacity expansion of a private line is avoided, the download pressure of a single download server is reduced, and the bandwidth utilization rate is improved. The file is transmitted by private network line strategy data, and the bandwidth is utilized when the service is idle, so that the utilization rate of the bandwidth is improved; the pressure on the bandwidth of the public network can be further reduced, and meanwhile, the security is higher, and the accuracy of data is ensured.
It should be understood that although the various steps in the flow diagrams of fig. 2-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least some of the steps in fig. 2-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 7, there is provided an edge cloud mirror image data processing apparatus including: an obtaining module 702, a duration determining module 704, a policy data determining module 706, and a data transmitting module 708, wherein:
an obtaining module 702, configured to obtain a service file to be transmitted and a mirror image data transmission policy dataset.
A duration determining module 704, configured to sequentially determine, according to the file size of the service file and the network line bandwidth of each mirror image data transmission policy data, a time-consuming duration required for transmitting the service file to an edge warehouse at an edge end.
A policy data determining module 706, configured to determine target mirror data transmission policy data from the mirror data transmission policy dataset according to each of the time-consuming durations.
A data transmission module 708, configured to transmit policy data based on the target mirror image data, and mirror the service file to an edge warehouse of the edge end by pushing.
In the edge cloud mirror image data processing device, a strategy data set is transmitted by acquiring a service file to be transmitted and mirror image data; sequentially determining time-consuming duration required for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration; transmitting strategy data based on target mirror image data, and pushing a service file to an edge warehouse of an edge end through a push mirror image; the target strategy data of the edge file transmission is determined according to the network line bandwidth and the size of the service file, so that the utilization rate of the network line bandwidth is improved, and the normal transmission of the service file is determined.
In one embodiment, an edge cloud mirroring data processing apparatus is provided, which includes, in addition to the obtaining module 702, the duration determining module 704, the policy data determining module 706 and the data transmitting module 708:
in one embodiment, the duration determining module 704 is further configured to determine a first time-consuming duration required for the service file to be transmitted from the central end to the edge warehouse of the edge end through the private network line according to the file size of the service file and the private network bandwidth of the private network line policy data.
In an embodiment, the duration determining module 704 is further configured to determine a second time-consuming duration required for the service file to be transmitted from the central end to the download server cluster through the public network line and to be transmitted to the edge warehouse at the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line policy data.
In one embodiment, the policy data determining module 706 is further configured to determine the private network line policy data as the target mirror data transmission policy data when the first elapsed time length is less than the second elapsed time length.
The data transmission module 708 is further configured to, based on the private network line policy data, trigger the idle time window of the edge terminal at a fixed time to mirror the service file to the edge warehouse of the edge terminal by pushing when the file size of the service file is within the first numerical region.
The obtaining module 702 is further configured to obtain a trigger time point of the timing trigger.
And the triggering module is used for triggering the idle time window of the edge end to push the business file to the edge warehouse of the edge end through a push mirror image when the triggering time point is in the time interval of the idle starting time point and the idle ending time point.
And the suspension module is used for suspending the timing trigger until the next idle starting time point when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, and executing the step of triggering the idle time window of the edge end to mirror the service file to the edge warehouse of the edge end through pushing.
In one embodiment, the policy data determining module 706 is further configured to determine that the public network line policy data is the target mirror data transmission policy data when the first time duration is longer than the second time duration.
And the encryption compression module is used for carrying out encryption compression processing on the service file to obtain a compressed file when the file size of the service file is in the second numerical value area based on the public network line strategy data.
And the address generation module is used for uploading the compressed file to the download server cluster through the public network entrance to generate a download address of the compressed file.
The triggering module is also used for sending the download address to the edge terminal, and the edge terminal is triggered to download the compressed file through the download address and conduct decompression and decryption processing to obtain the service file.
The data transmission module 708 is further configured to mirror the service file to an edge warehouse of the edge end through pushing.
And the encryption compression module is also used for carrying out encryption compression processing on the service file to obtain a compressed file when the file size of the service file is in a third numerical value area based on the public network line strategy data.
And the file splitting module is used for splitting the compressed file to obtain n sub-files.
And the address generation module is used for uploading the n sub-files to the download server cluster through the public network entrance to generate the download addresses of the sub-files.
The triggering module is also used for sending each download address to the edge terminal, and the edge terminal is triggered to download the subfiles through each download address and carry out decompression, decryption and combination processing to obtain the service files.
The data transmission module 708 is further configured to mirror the service file to an edge warehouse of the edge end through pushing.
In one embodiment, a central terminal acquires a service file to be transmitted and a mirror image data transmission strategy data set, and determines a first time-consuming duration required by the transmission of the service file from the central terminal to an edge warehouse of an edge terminal through a private network line; determining a second time-consuming duration required for transmitting the service file from the center end to the download server cluster through the public network line and transmitting the service file to the edge warehouse of the edge end through the download server, and acquiring a trigger time point of timing trigger when the first time-consuming duration is less than the second time-consuming duration and the file size of the service file is in a first numerical region; when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing; when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the idle time window of the triggering edge end again to push the service file to an edge warehouse of the edge end through a push mirror image; the bandwidth is utilized when the service is idle, so that the utilization rate of the bandwidth is improved; the pressure on the bandwidth of the public network can be further reduced, and meanwhile, the security is higher, and the accuracy of data is ensured.
When the first time consumption duration is longer than the second time consumption duration, based on the public network line strategy data, when the file size of the service file is in a second numerical value area, encrypting and compressing the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file; sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file; the business file is mirrored to an edge warehouse of an edge end through pushing; based on the public network line strategy data, when the file size of the service file is in a third numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file; splitting the compressed file to obtain n sub-files; uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files; sending each download address to an edge terminal, triggering the edge terminal to download the subfiles through each download address, decompressing, decrypting and combining to obtain a service file; the service files are pushed to the edge warehouse of the edge end in a mirror image mode, namely the files are encrypted and compressed, the security of the mirror image files in the internet and the utilization rate of network bandwidth are improved, the files are split and uploaded to a download server set to be distributed, the huge cost caused by the fact that the files are too large due to special line expansion is avoided, the download pressure of a single download server is reduced, and the bandwidth utilization rate is improved.
For specific definition of the edge cloud mirror image data processing device, reference may be made to the definition of the edge cloud mirror image data processing method in the foregoing, and details are not described herein again. The modules in the edge cloud mirror image data processing device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for business document data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an edge cloud mirroring data processing method.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory having a computer program stored therein and a processor that when executing the computer program performs the steps of:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse at an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting the strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a push mirror image.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
determining a first time-consuming duration required by the business file to be transmitted from a center end to an edge warehouse of an edge end through a private network line according to the file size of the business file and the private network bandwidth of the private network line strategy data;
and determining a second time-consuming duration required for transmitting the service file from the center end to the download server cluster through the public network line and transmitting the service file to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line strategy data.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
when the first time consumption duration is less than the second time consumption duration, determining the private network line strategy data as target mirror image data transmission strategy data;
transmitting strategy data based on target mirror image data, and mirroring a service file to an edge warehouse of an edge end by pushing a mirror image, wherein the method comprises the following steps:
based on the private network line strategy data, when the size of the business file is in the first numerical value area, the business file is pushed to an edge warehouse of the edge end through triggering an idle time window of the edge end at regular time.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a trigger time point of timing trigger;
and when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing an edge warehouse step of triggering an idle time window of the edge end to mirror the service file to the edge end through a push mirror.
In one embodiment, the processor when executing the computer program further performs the steps of:
when the first time consumption duration is longer than the second time consumption duration, determining that the public network line strategy data is the target mirror image data transmission strategy data;
transmitting strategy data based on target mirror image data, and mirroring a service file to an edge warehouse of an edge terminal by pushing a mirror image, wherein the method comprises the following steps:
based on the public network line strategy data, when the file size of the service file is in a second numerical value area, encrypting and compressing the service file to obtain a compressed file;
uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file;
sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file;
and pushing the service file to an edge warehouse of the edge end through a mirror image.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
based on the public network line strategy data, when the file size of the service file is in a third numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
splitting the compressed file to obtain n sub-files;
uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files;
sending each download address to the edge terminal, triggering the edge terminal to download the subfiles through each download address, decompressing, decrypting and combining to obtain a service file;
and pushing the service file to an edge warehouse of the edge end through a mirror image.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse at an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting the strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
determining a first time-consuming duration required by the business file to be transmitted from a center end to an edge warehouse of an edge end through a private network line according to the file size of the business file and the private network bandwidth of the private network line strategy data;
and determining a second time-consuming duration required for transmitting the service file from the center end to the download server cluster through the public network line and transmitting the service file to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line strategy data.
In one embodiment, the computer program when executed by the processor further performs the steps of:
when the first time consumption duration is less than the second time consumption duration, determining the private network line strategy data as target mirror image data transmission strategy data;
transmitting strategy data based on target mirror image data, and mirroring a service file to an edge warehouse of an edge terminal by pushing a mirror image, wherein the method comprises the following steps:
based on the private network line strategy data, when the size of the business file is in the first numerical value area, the business file is pushed to an edge warehouse of the edge end through triggering an idle time window of the edge end at regular time.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a trigger time point of timing trigger;
and when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to push the service file to an edge warehouse of the edge end through a push mirror image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the step of triggering the idle time window of the edge end to push the service file to the edge warehouse of the edge end through the push mirror image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
when the first time consumption duration is longer than the second time consumption duration, determining that the public network line strategy data is the target mirror image data transmission strategy data;
transmitting strategy data based on target mirror image data, and mirroring a service file to an edge warehouse of an edge end by pushing a mirror image, wherein the method comprises the following steps:
based on the public network line strategy data, when the file size of the service file is in a second numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
uploading the compressed file to a download server cluster through a public network entrance to generate a download address of the compressed file;
sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain a service file;
and pushing the service file to an edge warehouse of the edge end through a mirror image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
based on the public network line strategy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file;
splitting the compressed file to obtain n sub-files;
uploading the n sub-files to a download server cluster through a public network entrance, and generating download addresses of the sub-files;
sending each download address to an edge terminal, triggering the edge terminal to download the subfiles through each download address, decompressing, decrypting and combining to obtain a service file;
and pushing the service file to an edge warehouse of the edge end through a mirror image.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.

Claims (10)

1. An edge cloud mirror image data processing method, the method comprising:
acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
sequentially determining the time-consuming duration required for transmitting the service file to an edge warehouse at an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time-consuming duration;
and transmitting strategy data based on the target mirror image data, and pushing the service file to an edge warehouse of the edge end through a pushing mirror image.
2. The method according to claim 1, wherein the mirror image data transmission policy dataset includes private network line policy data and public network line policy data, and the determining a time-consuming duration required for transmitting the service file to an edge warehouse of an edge end in sequence according to a file size of the service file and a network line bandwidth of each of the mirror image data transmission policy data includes:
determining a first time-consuming duration required by the business file to be transmitted from a center end to an edge warehouse of the edge end through a private network line according to the file size of the business file and the private network bandwidth of the private network line strategy data;
and determining a second time-consuming duration required for the service file to be transmitted from the central end to the download server cluster through the public network line and to be transmitted to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line strategy data.
3. The method of claim 2, wherein determining target mirrored data transmission policy data from the mirrored data transmission policy dataset according to each of the time-consuming durations comprises:
when the first time-consuming duration is less than the second time-consuming duration, determining the private network line strategy data as target mirror image data transmission strategy data;
the transmitting the strategy data based on the target mirror image data and the mirror image of the service file to the edge warehouse of the edge end by pushing the mirror image comprises the following steps:
based on the private network line strategy data, when the size of the service file is in a first numerical value area, the service file is pushed to an edge warehouse of the edge end through triggering an idle time window of the edge end at regular time.
4. The method according to claim 3, wherein the idle time window has a corresponding idle start time point and idle end time point, and the mirroring of the service file to the edge repository of the edge end through the push by triggering the space time window of the edge end at regular time comprises:
acquiring a trigger time point of the timing trigger;
and when the triggering time point is in the time interval of the idle starting time point and the idle ending time point, triggering the idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing.
5. The method of claim 4, further comprising:
and when the triggering time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing triggering until the next idle starting time point, and executing the step of triggering the idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing.
6. The method of claim 2, wherein determining target mirrored data transmission policy data from the mirrored data transmission policy dataset according to each of the time-consuming durations comprises:
when the first time consumption duration is longer than the second time consumption duration, determining that the public network line strategy data is target mirror image data transmission strategy data;
the transmitting the strategy data based on the target mirror image data and the pushing the service file to the edge warehouse of the edge terminal by mirror image comprises the following steps:
based on the public network line strategy data, when the file size of the service file is in a second numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
uploading the compressed file to the download server cluster through a public network entrance, and generating a download address of the compressed file;
sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and carrying out decompression and decryption processing to obtain the service file;
and mirroring the service file to an edge warehouse of the edge end through pushing.
7. The method of claim 6, further comprising:
based on the public network line strategy data, when the file size of the service file is in a third numerical value area, carrying out encryption compression processing on the service file to obtain a compressed file;
splitting the compressed file to obtain n sub-files;
uploading the n sub-files to the download server cluster through a public network entrance, and generating download addresses of the sub-files;
sending each download address to the edge end, triggering the edge end to download the subfiles through each download address, decompressing, decrypting and combining to obtain the service file;
and mirroring the service file to an edge warehouse of the edge end through pushing.
8. An edge cloud image data processing apparatus, the apparatus comprising:
the acquisition module is used for acquiring a service file to be transmitted and a mirror image data transmission strategy data set;
the time length determining module is used for sequentially determining the time length consumed for transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;
a policy data determining module, configured to determine target mirror data transmission policy data from the mirror data transmission policy data set according to each time-consuming duration;
and the data transmission module is used for transmitting the strategy data based on the target mirror image data and transmitting the service file to the edge warehouse of the edge end through a push mirror image.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202110384026.0A 2021-04-09 2021-04-09 Edge cloud mirror image data processing method and device, computer equipment and storage medium Pending CN115208872A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110384026.0A CN115208872A (en) 2021-04-09 2021-04-09 Edge cloud mirror image data processing method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110384026.0A CN115208872A (en) 2021-04-09 2021-04-09 Edge cloud mirror image data processing method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115208872A true CN115208872A (en) 2022-10-18

Family

ID=83571553

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110384026.0A Pending CN115208872A (en) 2021-04-09 2021-04-09 Edge cloud mirror image data processing method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115208872A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115904719A (en) * 2022-12-02 2023-04-04 杭州义益钛迪信息技术有限公司 Data acquisition method and device, electronic equipment and storage medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105205143A (en) * 2015-09-18 2015-12-30 杭州华为数字技术有限公司 File storage and processing method, device and system
US20170149688A1 (en) * 2015-11-25 2017-05-25 International Business Machines Corporation Configuring resources to exploit elastic network capability
CN108401031A (en) * 2018-03-13 2018-08-14 深圳市腾讯计算机系统有限公司 A kind of image file transmission method, device and storage medium
CN109076357A (en) * 2016-05-28 2018-12-21 华为技术有限公司 Application method, relevant device and system are migrated in mobile limbic system
CN110896404A (en) * 2018-09-12 2020-03-20 华为技术有限公司 Data processing method and device and computing node
CN110958138A (en) * 2019-11-14 2020-04-03 新华三大数据技术有限公司 Container expansion method and device
CN111131379A (en) * 2019-11-08 2020-05-08 西安电子科技大学 Distributed flow acquisition system and edge calculation method
CN111786901A (en) * 2020-05-21 2020-10-16 网宿科技股份有限公司 Transmission parameter self-adaptive adjusting method and acceleration service system
CN111800442A (en) * 2019-04-08 2020-10-20 阿里巴巴集团控股有限公司 Network system, mirror image management method, device and storage medium
CN112565325A (en) * 2019-09-26 2021-03-26 华为技术有限公司 Mirror image file management method, device and system, computer equipment and storage medium

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105205143A (en) * 2015-09-18 2015-12-30 杭州华为数字技术有限公司 File storage and processing method, device and system
US20170149688A1 (en) * 2015-11-25 2017-05-25 International Business Machines Corporation Configuring resources to exploit elastic network capability
US20180083888A1 (en) * 2015-11-25 2018-03-22 International Business Machines Corporation Configuring resources to exploit elastic network capability
CN109076357A (en) * 2016-05-28 2018-12-21 华为技术有限公司 Application method, relevant device and system are migrated in mobile limbic system
CN108401031A (en) * 2018-03-13 2018-08-14 深圳市腾讯计算机系统有限公司 A kind of image file transmission method, device and storage medium
CN110896404A (en) * 2018-09-12 2020-03-20 华为技术有限公司 Data processing method and device and computing node
CN111800442A (en) * 2019-04-08 2020-10-20 阿里巴巴集团控股有限公司 Network system, mirror image management method, device and storage medium
CN112565325A (en) * 2019-09-26 2021-03-26 华为技术有限公司 Mirror image file management method, device and system, computer equipment and storage medium
CN111131379A (en) * 2019-11-08 2020-05-08 西安电子科技大学 Distributed flow acquisition system and edge calculation method
CN110958138A (en) * 2019-11-14 2020-04-03 新华三大数据技术有限公司 Container expansion method and device
CN111786901A (en) * 2020-05-21 2020-10-16 网宿科技股份有限公司 Transmission parameter self-adaptive adjusting method and acceleration service system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李云昊: "边缘计算模式下实时数据处理应用执行优化系统设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》, 15 June 2020 (2020-06-15) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115904719A (en) * 2022-12-02 2023-04-04 杭州义益钛迪信息技术有限公司 Data acquisition method and device, electronic equipment and storage medium
CN115904719B (en) * 2022-12-02 2023-12-08 杭州义益钛迪信息技术有限公司 Data acquisition method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN107819828B (en) Data transmission method and device, computer equipment and storage medium
Xiang et al. Joint latency and cost optimization for erasurecoded data center storage
US11442790B2 (en) Resource scheduling system, method and server for ensuring high availability of services
CN109040252B (en) File transmission method, system, computer device and storage medium
US20180240114A1 (en) Transaction verification in a consensus network
WO2020133967A1 (en) Method for scheduling shared computing resources, shared computing system, server, and storage medium
CN102790760B (en) Data synchronization method based on directory tree in safe network disc system
CN111555957B (en) Kafka-based synchronous message service system and implementation method
WO2016029744A1 (en) Metadata recovery method and relevant device
US20140214782A1 (en) Distributed Storage Object Delete
CN103986764A (en) Equipment and method used for multi-client collaborative file uploading
CN113411404A (en) File downloading method, device, server and storage medium
US20110154015A1 (en) Method For Segmenting A Data File, Storing The File In A Separate Location, And Recreating The File
CN114546681A (en) Kafka-based message processing method, device, equipment and storage medium
CN104219298A (en) Cluster system and data backup method thereof
CN110267060B (en) Video file storage injection method and device, computer equipment and storage medium
CN115208872A (en) Edge cloud mirror image data processing method and device, computer equipment and storage medium
CN111416992A (en) File transcoding method and device
CN114048201A (en) Distributed stream computing engine Flink-based key field real-time deduplication method
US9245048B1 (en) Parallel sort with a ranged, partitioned key-value store in a high perfomance computing environment
US10250515B2 (en) Method and device for forwarding data messages
CN117331576A (en) Application pre-download method, device, computer and storage medium
CN111061682B (en) Data caching method, reading method, electronic equipment and storage medium
CN113742681B (en) Account management method and device, computer equipment and storage medium
WO2023287357A2 (en) Data transmission method, node and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination