CN116610643B - Method, device and program product for monitoring directory compression downloading process - Google Patents

Method, device and program product for monitoring directory compression downloading process Download PDF

Info

Publication number
CN116610643B
CN116610643B CN202310876743.4A CN202310876743A CN116610643B CN 116610643 B CN116610643 B CN 116610643B CN 202310876743 A CN202310876743 A CN 202310876743A CN 116610643 B CN116610643 B CN 116610643B
Authority
CN
China
Prior art keywords
compression
file
directory
list
compressed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310876743.4A
Other languages
Chinese (zh)
Other versions
CN116610643A (en
Inventor
刘健
李欣然
孙一飞
杨艳光
马辉
张玉鑫
卞永锋
张同虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CCB Finetech Co Ltd
Original Assignee
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CCB Finetech Co Ltd filed Critical CCB Finetech Co Ltd
Priority to CN202310876743.4A priority Critical patent/CN116610643B/en
Publication of CN116610643A publication Critical patent/CN116610643A/en
Application granted granted Critical
Publication of CN116610643B publication Critical patent/CN116610643B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application provides a monitoring method, a device and a program product for a directory compression downloading process. The technical field of big data processing comprises the steps of receiving an aggregation compression request sent by a client; when the method is in a complete catalog downloading mode, generating a file list to be compressed and an empty catalog list according to the request catalog list, and monitoring the compression downloading process of the file list to be compressed and the empty catalog list; and when the file list is in the resource downloading mode, generating a file list to be compressed according to the request directory list, and monitoring the compression downloading process of the file list to be compressed. According to different types of compression downloading modes, different forms of directory compression downloading are adopted, the directory monitors the compression downloading process of the directory in real time in the compression downloading process, and compression downloading progress information obtained by monitoring is fed back to the client so as to monitor the compression downloading process of the request directory list in real time.

Description

Method, device and program product for monitoring directory compression downloading process
Technical Field
The present application relates to the field of big data processing technologies, and in particular, to a method, an apparatus, and a program product for monitoring a directory compression and downloading process.
Background
In the prior art, when downloading a file, a file link is generally loaded on a webpage, and a browser responds to a downloading request of a user and initiates a request to a server where the file is located so as to acquire a file text for downloading.
However, in some scenarios, when a user needs to compress a plurality of remote websites into a single zip compressed file for overall downloading, the process of downloading from the directory level cannot be realized, and the downloading process of the directory level is monitored dynamically in real time. Especially when there is an empty directory under the download path, the integrity of the directory structure is also affected during the directory compression download process.
Disclosure of Invention
The application provides a monitoring method, a device and a program product for a directory compression and downloading process, which are used for maintaining the integrity of a directory structure in the directory compression and downloading process.
In a first aspect, the present application provides a method for monitoring a directory compression download process, which is applied to a file compression server, and includes:
receiving an aggregation compression request sent by a client, wherein the aggregation compression request comprises a request directory list and a compression downloading mode;
when the compression downloading mode is determined to be a complete catalog downloading mode, generating a file list to be compressed and an empty catalog list according to the request catalog list, and monitoring compression downloading processes of the file list to be compressed and the empty catalog list to obtain compression downloading progress information in the complete catalog downloading mode;
When the compression downloading mode is determined to be a resource downloading mode, generating a file list to be compressed according to the request directory list, and monitoring the compression downloading process of the file list to be compressed to acquire compression downloading progress information in the resource downloading mode;
and when a progress monitoring query request sent by the client is received, the compressed downloading progress information matched with the compressed downloading mode is sent to the client.
In a second aspect, the present application provides a monitoring device for a directory compression downloading process, including:
the aggregation compression request receiving module is used for receiving an aggregation compression request sent by the client, wherein the aggregation compression request comprises a request catalog list and a compression downloading mode;
the first compression downloading progress information acquisition module is used for generating a file list to be compressed and an empty directory list according to the request directory list when the compression downloading mode is determined to be a complete directory downloading mode, and monitoring the compression downloading progress of the file list to be compressed and the empty directory list to acquire compression downloading progress information in the complete directory downloading mode;
The second compression downloading progress information acquisition module is used for generating a file list to be compressed according to the request directory list when the compression downloading mode is determined to be a resource downloading mode, and monitoring the compression downloading progress of the file list to be compressed to acquire the compression downloading progress information in the resource downloading mode;
and the monitoring module is used for sending the compression downloading progress information matched with the compression downloading mode to the client when receiving the progress monitoring query request sent by the client.
In a third aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored in the memory to implement the method according to the present application.
In a fourth aspect, the present application provides a computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of the present application.
In a fifth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of the application.
According to the method, different forms of directory compression downloading are adopted according to different types of compression downloading modes, the compression downloading process of the directory is monitored in real time in the compression downloading process, and compression downloading progress information obtained by monitoring is fed back to the client, so that the compression downloading process of the request directory list is monitored in real time.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart of a monitoring method for a directory compression download process according to a first embodiment of the present application;
fig. 2 is a flowchart of a monitoring method for a directory compression downloading process according to a second embodiment of the present application;
FIG. 3 is a directory interaction timing chart of a directory compression download in a complete directory download mode according to a second embodiment of the present application;
fig. 4 is a flowchart of a monitoring method of a directory compression downloading process according to a third embodiment of the present application;
FIG. 5 is an interactive timing chart of directory compression download in resource download mode according to the third embodiment of the present application;
fig. 6 is a schematic structural diagram of a monitoring device for a directory compression downloading process according to a third embodiment of the present application;
Fig. 7 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings. The technical scheme of the application obtains, stores, uses, processes and the like the data, which all meet the relevant regulations of national laws and regulations.
Example 1
Fig. 1 is a flowchart of a monitoring method of a directory compression downloading process according to an embodiment of the present application, as shown in fig. 1, the method includes the following steps:
step S101, an aggregate compression request sent by a client is received.
In particular, the client may be a web application, and in this embodiment, the specific type of the client is not limited, and the client may respond to the upper-layer service selected by the user, for example, a test report list, to display the downloadable catalogue and the option of the compressed download mode. The user can select from the displayed downloadable catalogs, select a plurality of catalogs to be downloaded, determine a compression downloading mode according to the options, generate a request catalogue list based on the catalogs selected by the user, and generate an aggregate compression request according to the request catalogue list and the selected compression downloading mode. The compression download mode in this embodiment may include a complete directory download mode or a resource download mode, where the resource download mode refers to downloading only the effective resources in the directory; the complete catalog downloading mode refers to downloading the effective resources in the catalog and simultaneously downloading the empty catalog which does not contain the effective resources so as to ensure the integrity of the catalog structure. In this embodiment, the aggregate compression request includes not only the request directory list but also the download request global unique ID, and the download request global unique ID is used to identify the aggregate compression request.
Optionally, the file compression server includes a compression module and a monitoring module, and after receiving the aggregate compression request sent by the client, the file compression server further includes: creating a compression instance and a monitoring object of the compression instance by the catalog through the compression module; binding the output stream of the compression instance with the monitoring object, and communicating the monitoring object with the response output stream of the aggregation compression request.
Specifically, the file compression server includes a compression module, a monitoring module and a storage module, after receiving an aggregate compression request sent by a client, a monitoring object MonitorOutputStream is created for the aggregate compression request through the monitoring module, and the monitoring object monitoroutputstreamis bound to a response output stream of the aggregate compression request, in addition, a compression instance ziploutsteam is created through the compression module, and the compression instance ziploutputsteam is bound to the monitoring object MonitorOutputStream, so that the compression instance ziplotputsteam and the response output stream are communicated through the monitoring object MonitorOutputStream, and compressed data transmitted to the response output stream through the compression instance can be monitored by the monitoring object MonitorOutputStream, so that the monitoring object monitoroutputstreamin the embodiment is a transit object, and in the transit process, the total number of bytes flowing through itself, the initial byte initiation time stamp, the number of bytes flowing through a specified period and the like can be recorded.
Step S102, when the compression download mode is determined to be the complete catalog download mode, generating a file list to be compressed and an empty catalog list according to the request catalog list, and monitoring the compression download progress of the file list to be compressed and the empty catalog list to obtain compression download progress information in the complete catalog download mode.
Optionally, generating the file list to be compressed and the empty directory list according to the request directory list includes: and recursively traversing the request directory and the multi-level subdirectories of the request directory to obtain all nodes, wherein the nodes comprise the directory and the effective files. Determining an empty directory in the node, wherein the empty directory is a directory in the node which does not contain any child node; and constructing a file list to be compressed according to all the effective files, and constructing an empty directory list according to the empty directory.
Optionally, constructing an empty directory list according to the empty directory includes: acquiring a preset arrangement rule; and ordering the empty catalogues according to the arrangement rule to construct an empty catalogue list.
Specifically, in this embodiment, since the directories are mainly directed to the resource server, and the file compression server is communicatively connected to the resource server, when it is determined that the current compression download mode is the complete directory download mode, the file compression server may access the structure information of the directories in the resource server, traverse each directory in the request directory list, and determine whether the sub-node is included in each directory according to the access result, where a node may refer to a directory and an active file, that is, when it is determined that three main directories, namely, directory 1, directory 2, and directory 3 are included in the request directory list, it is determined that 2 active files are included in the directory 1, and one sub-directory 2.1 is included in the directory 2, but no active file is included in the sub-directory 2.1, and no active file is included in the directory 3, and the above sub-directories 2.1 and 3 are referred to as empty directories, and the above-mentioned sub-directories 1 and 2 are referred to as non-empty directories.
In this embodiment, after determining the type of the directory, an effective file may be included in the non-empty directory, but no effective file is included in the empty directory, so in this embodiment, a file list to be compressed is constructed according to all the effective files included in the non-empty directory, and an empty directory list is constructed according to the empty directory, where an effective file refers to a file that is included in the non-empty directory and is capable of being effectively compressed and downloaded. The file compression server obtains access right information for each non-empty directory according to the pre-configuration information, and the file compression server is in communication connection with the resource server, so that when the file compression server accesses the structure information of the directory in the resource server, the file compression server also obtains attribute information of the files under each non-empty directory, and the attribute information not only comprises file identifiers but also comprises file sizes. And constructing a file list to be compressed according to the acquired attribute information of the files, so that the acquired file list to be compressed not only contains file identifiers and total number of the files, but also contains the sizes of the files and the total size of all the files. The file size contained in the file list to be compressed and the indirectly acquired total file size can be used for facilitating progress monitoring from multiple dimensions such as the number of files, the number of bytes and the like in the subsequent compression and downloading process.
In addition, after the empty catalogs are determined by accessing the catalogue structure of the resource server, the acquired empty catalogues are ordered according to a preset arrangement rule to construct an empty catalogue list. The preset arrangement rule may be arranged according to the order of the creation date of the empty directory in the resource server from early to late, or may be arranged according to the order of the empty directory types identified, and in this embodiment, the specific form of the arrangement rule is not limited, and it is within the scope of the present application as long as the empty directory list can be constructed based on the determined empty directory.
Optionally, monitoring the compression download process of the file list to be compressed and the empty directory list to obtain compression download progress information in the complete directory download mode, including: file resources corresponding to the file list to be compressed are called from the resource server, and the file resources and each empty directory in the empty directory list are compressed and downloaded; and monitoring the compression downloading process of each file resource and each empty directory in the empty directory list to obtain compression progress information in a complete directory downloading mode.
Step S103, when the compression download mode is determined to be the resource download mode, generating a file list to be compressed according to the request directory list, and monitoring the compression download progress of the file list to be compressed to obtain compression download progress information in the resource download mode.
Optionally, generating the file list to be compressed according to the request directory list includes: recursively traversing the request directory and the multi-level subdirectories of the request directory to obtain all nodes, wherein the nodes comprise the directory and the effective file; determining valid files in all nodes; and constructing a file list to be compressed according to the directory effective file.
Optionally, monitoring a compression download process of the file list to be compressed to obtain compression download progress information in a resource download mode, including: file resources corresponding to the file list to be compressed are called from a resource server, and the file resources are compressed and downloaded; and monitoring the compression downloading process of the file resource to obtain compression progress information in a resource downloading mode.
Specifically, when the current compression download mode is determined to be the resource download mode, the compression download is performed only for the effective files in each directory, and because the file compression server is in communication with the resource server, the file compression server can access the structure information of the directory in the resource server, and traverse each directory in the directory list to reserve the effective files in the non-empty directory, and access right information for each non-empty directory is acquired according to the pre-configuration information. And constructing a file list to be compressed according to the acquired attribute information of the files, so that the acquired file list to be compressed not only contains file identifiers and total number of files, but also contains the sizes of all the files and the total size of all the files. The file size contained in the file list to be compressed and the indirectly acquired total file size can be used for facilitating progress monitoring from multiple dimensions such as the number of files, the number of bytes and the like in the subsequent compression and downloading process.
It should be noted that, in the resource download mode, since only the effective file is compressed and downloaded, and the empty directory is not considered, only the file list to be compressed including the effective file is generated according to the request directory list in the resource download mode, and the empty directory list is not required to be generated. And the final compression downloading result only contains the file resources corresponding to each effective file in the request directory list.
Step S104, when receiving the progress monitoring query request sent by the client, the compressed downloading progress information matched with the compressed downloading mode is sent to the client.
Optionally, the sending the compressed download progress information matched with the compressed download mode to the client includes: extracting a request monitoring type from the progress monitoring query request; and calling target compression downloading progress information corresponding to the request monitoring type, and sending the target compression downloading progress information to the client so as to display the target compression downloading progress information on the client.
Specifically, after the compression downloading process of each file resource is monitored in real time to obtain the compression downloading progress information, the client may send a progress monitoring query request to the file compression server through the progress query interface, where the progress monitoring query request may be generated by user triggering or may be periodically generated by the client according to the configuration information.
After receiving the progress query request, the file compression server in this embodiment extracts the request global unique identifier ID from the progress query request, determines to which aggregate compression request the user needs to query, and extracts the request monitoring type from the progress monitoring query request. The request monitoring type in this embodiment may be a data name in the overall compression downloading progress information, or may be a data name in the sub-compression downloading progress information, so long as the data collected in the compression downloading process can be queried by a user, and the specific content of the request monitoring type is not limited in this embodiment.
When the file compression server determines the request monitoring type, the file compression server can call target compression progress information corresponding to the request monitoring type from the storage module and send the target compression downloading progress information to the client. For example, the target compression progress information may be 30% of the total byte number compression progress, 25% of the total file number compression progress, and 3 of the current completed compressed file number, which is, of course, merely illustrative, and the specific content of the target compression download progress information is not limited in this embodiment.
It should be noted that, after the file compression server sends the target compression download progress information to the client, the client receives the target compression download progress information through the progress query interface, and further displays a compression download completion prompt, for example, "may you be happy and you download complete" when the compression download process is completed. In the embodiment, the universal resource downloading monitoring device is combined with the file management device, and the client can query the downloading progress in parallel through the processes except the downloading process, so that real-time monitoring in the directory downloading process is realized.
The method and the device acquire the file resources corresponding to the file list to be compressed and the empty directory list associated with the directory list to be requested, simultaneously compress and download the file resources and all the empty directories contained in the empty directory list, monitor the compression and download process of the directory in real time in the compression and download process, and feed back the compression and download progress information acquired by monitoring to the client so as to monitor the compression and download process of the directory list to be requested in real time.
Example two
Fig. 2 is a flowchart of a monitoring method for a directory compression downloading process according to an embodiment of the present application, where, based on the foregoing embodiment, in a complete directory downloading mode, a process of monitoring a compression downloading process of a file list to be compressed and an empty directory list to obtain compression downloading progress information in the complete directory downloading mode is specifically described, and as shown in fig. 2, the method includes the following steps:
Step S201, file resources corresponding to the file list to be compressed are called from the resource server, and the file resources and each empty directory in the empty directory list are compressed and downloaded.
In the embodiment, the process of compressing and downloading the directory in the complete directory downloading mode is mainly described.
Optionally, the file compression server includes a compression module and a monitoring module, and before the file resource corresponding to the file list to be compressed is called to the resource server, the file compression server further includes: creating a monitoring object aiming at the file list to be compressed and the empty directory list through a monitoring module, and binding the monitoring object to a response output stream; and creating a compression instance for the file list to be compressed and the empty directory list through the compression module, and binding the compression instance with the monitoring object so as to communicate the compression instance with the response output through the monitoring object.
Specifically, the file compression server includes a compression module, a monitoring module and a storage module, after obtaining a file list to be compressed and an empty directory list, the file list to be compressed and the empty directory list are stored in the storage module, a monitoring object monitoroutputstreamis created for the two lists through the monitoring module, and the monitoring object monitoroutputstreamis bound to a response output stream of an aggregate compression request, in addition, a compression instance ziplotputstepsteptom is created for the two lists through the compression module, and the compression instance ziplotputsteptom is bound to the monitoring object monitoroutputstepstreamso as to communicate the compression instance ziplottstepsteptom and the response output stream through the monitoring object MonitorOutputStream, and compressed data transmitted to the response output stream through the compression instance can be monitored by the monitoring object MonitorOutputStream, so that the monitoring object monitorstreamin the embodiment is a transit object, and the total number of bytes flowing through the transit in the transit process itself can be recorded, and the number of bytes flowing through the time stamp is specified in the transit time period.
Optionally, the method includes the steps of calling the file resource corresponding to the file list to be compressed from the resource server, and compressing and downloading the file resource and each empty directory in the empty directory list, including: creating a file compression class of each effective file in a file list to be compressed by running a compression example, and calling file resources of each effective file to a resource server, wherein the file compression class contains attribute information of the file; compressing the file compression class and the file resource to obtain a file compression downloading result; creating a catalog compression class of each empty catalog in the empty catalog list by running a compression example, wherein the catalog compression class contains attribute information of the empty catalog; and compressing the catalog compression class to obtain an empty catalog compression downloading result.
Specifically, in this embodiment, for a file list to be compressed determined by an active file under a non-empty directory, not only a corresponding file compression class needs to be created for each active file to store attribute information of the file, but also file resources of each active file are called to a resource server, so that for the non-empty directory, specifically, the file compression class and the file resources are compressed simultaneously, and thus a file compression and downloading result including both file data and file attribute information is obtained. While for the empty directory list, a corresponding directory compression class is created for each empty directory, but since the files are not contained under each empty directory, the corresponding resources do not need to be called to a resource server, and therefore, only the directory compression class is compressed and downloaded for the empty directory.
Optionally, creating a file compression class of each effective file in the file list to be compressed by running the compression instance, and calling the file resource of each effective file to the resource server, including: traversing the file list to be compressed by running a compression example; when the target file which is not downloaded in a compression mode is contained in the file list to be compressed according to the traversing result, creating a file compression class for the target file, and storing attribute information of the target file into the file compression class; and sending a resource reading request for the target file to a resource server, receiving the target file resource fed back by the resource server based on the resource reading request, and writing the target file resource into the compression instance in the form of byte stream.
Optionally, after compressing the file compression class and the file resource to obtain a file compression downloading result, the method further includes: and carrying out forced refreshing treatment on the output buffer memory of the compression example to ensure that all the non-empty directory downloading compression results are transmitted to the response output stream.
In a specific implementation, after creating a compression instance ZipOutputSteam and a monitor object MonitorOutputStream for a file list to be compressed, a zip compression byte stream is generated by running the compression instance ZipOutputSteam, and an output target of zip is directed to MonitorOutputStream for realizing interception monitoring of an output stream. Traversing the file list to be compressed by operating the compression module, judging whether the file list to be compressed contains an uncompleted task, namely an uncompressed downloaded target file, creating a file compression class ZipEntry1 for the uncompressed downloaded target file when the uncompressed downloaded target file exists, wherein ZipEntry1 is an object for the target file, and storing attribute information of the target file into ZipEntry 1. The file compression server initiates a resource reading request according to a downloading protocol, a downloading address and access authority information of a resource, the resource reading request comprises an identification of a target file, when the corresponding resource server receives the resource reading request, the target file resource is fed back to the file compression server, and the file compression server writes the target file into a compression instance in a byte stream mode. At this time, the compression instance ziploutstem includes the target file resource obtained by the target file in the byte stream form, and also includes the attribute information of the target file in the file compression class ZipEntry1, and the compression instance ziploutputstem compresses the file compression class and the target file resource to obtain the file compression downloading result. When the compression of the target file is determined to be completed, the compression instance ZipOutputSteam is subjected to forced refreshing processing so as to ensure that the compressed file is completely transmitted to the output stream of the response.
Optionally, creating a directory compression class of each empty directory in the empty directory list by running a compression instance includes: traversing the empty directory list by running the compression instance; when the target empty catalogue which is not downloaded in a compression mode is determined to be contained in the empty catalogue list according to the traversing result, creating a catalogue compression class for the target empty catalogue; and saving the attribute information of the target empty directory into a directory compression class.
Optionally, after compressing the directory compression class to obtain the empty directory compression download result, the method further includes: the output buffer memory of the compression example is subjected to forced refreshing treatment so as to ensure that the whole transmission of the compression downloading result of the empty directory to the response output stream is ensured; and closing the compression instance when the fact that all the empty directories in the empty directory list are compressed is determined to be completed.
In another specific implementation, by running the compression instance ZipOutputSteam to traverse the empty directory list, judging whether an unfinished task, namely an uncompressed downloaded empty directory, is contained in the empty directory list, creating a directory compression class ZipEntry2 for the uncompressed downloaded target empty directory when the existence of the unfinished task is determined, and storing attribute information of the target directory into ZipEntry2, wherein ZipEntry2 is an object for the directory, and the directory type object path is marked with "/". At this time, the compression instance ziploutstem contains attribute information of the target empty directory in the directory compression class ZipEntry2, and the compression instance ziploutstem compresses the directory compression class to obtain a directory compression downloading result. And when the fact that the task to be handled does not exist in the empty directory list is determined, closing the compressed instance ZipOutputSteam.
In this embodiment, after the file compression server completes the compression of all files in the file list to be compressed and all empty directories in the empty directory list, the compressed files are fed back to the client that sends the aggregate compression request by responding to the output stream. The client determines the disk file according to the local storage target path selected by the user, and stores the compressed file in the disk file selected by the user in advance, so that the empty catalogs and the non-empty catalogs in the request catalog list are stored in the disk file of the client in a binary stream mode.
Step S202, the compression downloading process of each file resource and each empty directory in the empty directory list is monitored, and compression progress information in a complete directory downloading mode is obtained.
Optionally, monitoring a compression downloading process of each file resource and each empty directory in the empty directory list to obtain compression progress information in a complete directory downloading mode, including: monitoring the output of the compression instance through a monitoring object in the monitoring module to obtain the number of bytes currently compressed and the final generation time; and monitoring the operation of the compression instance through a monitoring module, and acquiring the integral compression downloading progress, integral compression weighting progress and sub-compression downloading progress information of each effective file and each empty directory aiming at the file list to be compressed and the empty directory list in a complete directory downloading mode.
Optionally, the overall compression downloading progress information includes: start writing time, compression completion time, current written byte number and last writing time, current completed compressed file number, current completed compressed directory number, total byte number compression progress and total file number compression progress; the sub-compression download progress information includes: the start writing time of each file, the end time of each file, the number of bytes written in each file, the start writing time of each empty directory, the end time of each empty directory and the number of bytes written in each empty directory; the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the sum of the current completed compressed file number and the current completed compressed directory number to the sum of the total file number and the total empty directory number.
Specifically, in this embodiment, the monitor object monitor output stream in the monitor module can only monitor the output of the compression instance to obtain the number of currently compressed generated bytes and the last generation time, and uses the data obtained by the monitor object as a part of the content of the overall compression downloading progress information. Meanwhile, the monitoring module monitors the running process of the compression instance to acquire other contents of the overall downloading progress information of the file list to be compressed and the empty directory list, such as the start writing time, the compression completion time, the total number of bytes written currently and the last writing time, the number of files compressed currently, the number of directories compressed currently, the total number of bytes compressed progress and the total number of files compressed progress; the specific compression downloading progress of each file in the file list to be compressed is also monitored correspondingly, and subcompression downloading progress information corresponding to each file is obtained, such as the starting writing time of each file, the ending time of each file and the written byte number of each file; the specific compression progress of each empty directory in the empty directory list is also monitored correspondingly, and sub-compression downloading progress information corresponding to each empty directory is obtained, for example, when each empty directory starts to be written, the ending time of each space directory and the written byte number of each empty directory.
Wherein, the overall compression progress information not only contains the individual progress of byte number and file number, but also contains the total byte number of the file and the overall compression weighted progress of the total compression item of the file. For example, if the first weight for the byte compression schedule is 0.9 and the second weight for the file compression schedule is 0.1, the overall compression weighted schedule can be calculated using the following formula (1): overall compression weighted progress = total byte compression progress 0.9+ total file number compression progress 0.1 = current compression byte number/total byte number 0.9+ current compression (file number + empty directory number)/total (file number + empty directory number) 0.1 (1)
For example, when it is determined that the current number of compressed words is 1500, the current number of compressed files is 3, the current number of compressed empty directories is 1, the total number of bytes is 3000, the total number of files is 6, and the total number of empty directories is 2, the above values are substituted into formula (1), so that the overall compression weighting progress is 0.5.
In this embodiment, after the compression download progress information is obtained through monitoring, the obtained compression download progress information is also saved, and when a progress monitoring query request sent by the client is received, the compression download progress information is sent to the client.
The method and the system carry out catalog compression downloading in a complete catalog downloading mode, monitor the compression downloading process of the catalog in real time in the compression downloading process, and feed back the compression downloading progress information obtained by monitoring to the client so as to monitor the compression downloading process of the request catalog list in real time.
Example III
Fig. 4 is a flowchart of a method for monitoring a directory compression download process according to an embodiment of the present application, where based on the above embodiment, in a resource download mode, a compression download progress of a file list to be compressed is monitored to obtain compression download progress information in the resource download mode, and as shown in fig. 4, the method includes the following steps:
step 301, file resources corresponding to the file list to be compressed are called from the resource server, and the file resources are compressed and downloaded.
In the embodiment, the process of compressing and downloading the directory in the resource downloading mode is mainly described.
Specifically, after the file list to be compressed is obtained in the resource downloading mode, the file list to be compressed is stored in the storage module, a monitoring object monitorOutputStream is created for the file list to be compressed through the monitoring module, the monitoring object monitorOutputStream is bound to the response output stream, in addition, a compression instance ZipOutputSteam is created for the file list to be compressed through the compression module, the compression instance ZipOutputSteam is bound to the monitoring object monitorOutputStream, so that the compression instance ZipOutputPuttStream is communicated with the response output stream through the monitoring object monitorOutputStream, compressed data transmitted to the response output stream through the compression instance can be monitored by the monitoring object monitorOutputStream, and therefore, the monitoring object monitorOutputStream in the embodiment is a transit object, in the transit process, the total number of bytes flowing through the device can be recorded, the initial byte initiating time stamp and the number of bytes flowing through the device in a specified acquisition period can be recorded.
Optionally, the method includes the steps of calling the file resource corresponding to the file list to be compressed from the resource server, and compressing and downloading the file resource, including: creating a file compression class of each effective file in a file list to be compressed by running a compression example, and calling file resources of each effective file to a resource server, wherein the file compression class contains attribute information of the file; and compressing the file compression class and the file resource to obtain a file compression downloading result.
Optionally, creating a file compression class of each effective file in the file list to be compressed by running the compression instance, and calling the file resource of each effective file to the resource server, including: traversing the file list to be compressed by running a compression example; when the target file which is not downloaded in a compression mode is contained in the file list to be compressed according to the traversing result, creating a file compression class for the target file, and storing attribute information of the target file into the file compression class; and sending a resource reading request for the target file to a resource server, receiving the target file resource fed back by the resource server based on the resource reading request, and writing the target file resource into the compression instance in the form of byte stream.
Optionally, after compressing the file compression class and the file resource to obtain a file compression downloading result, the method further includes: and carrying out forced refreshing treatment on the output buffer memory of the compression example to ensure that all the non-empty directory downloading compression results are transmitted to the response output stream.
In a specific implementation, after creating a compression instance ZipOutputSteam and a monitor object MonitorOutputStream for a file list to be compressed, a zip compression byte stream is generated by running the compression instance ZipOutputSteam, and an output target of zip is directed to MonitorOutputStream for realizing interception monitoring of an output stream. Traversing the file list to be compressed by operating the compression module, judging whether the file list to be compressed contains an uncompleted task, namely an uncompressed downloaded target file, creating a file compression class ZipEntry1 for the uncompressed downloaded target file when the uncompressed downloaded target file exists, and storing attribute information of the target file into the ZipEntry 1. The file compression server initiates a resource reading request according to a downloading protocol, a downloading address and access authority information of a resource, the resource reading request comprises an identification of a target file, when the corresponding resource server receives the resource reading request, the target file resource is fed back to the file compression server, and the file compression server writes the target file into a compression instance in a byte stream mode. At this time, the compression instance ziploutstem includes the target file resource obtained by the target file in the byte stream form, and also includes the attribute information of the target file in the file compression class ZipEntry1, and the compression instance ziploutputstem compresses the file compression class and the target file resource to obtain the file compression downloading result. When the compression of the target file is determined to be completed, the compression instance ZipOutputSteam is subjected to forced refreshing processing so as to ensure that the compressed file is completely transmitted to the output stream of the response.
It should be noted that, in the resource download mode, an empty directory list is not involved, so that when the directory is compressed, only the file resource corresponding to the file list to be compressed needs to be compressed and downloaded, but the compression and downloading process for the empty directory is not involved, and the creation process of the directory compression class is not involved. Therefore, only file resources corresponding to the effective files are contained in the finally obtained compression downloading result.
Step 302, monitoring the compression downloading process of the file resource, and obtaining compression progress information in a resource downloading mode.
Optionally, monitoring a compression downloading process of the file resource to obtain compression progress information in a resource downloading mode, including: monitoring the output of the compression instance through a monitoring object in the monitoring module to obtain the number of bytes currently compressed and the final generation time; and monitoring the operation of the compression instance through a monitoring module, and acquiring the overall compression downloading progress, the overall compression weighting progress and the sub-compression downloading progress information of each effective file aiming at the file list to be compressed in a resource downloading mode.
Optionally, the overall compression downloading progress information includes: start writing time, compression completion time, number of bytes currently written and last writing time, number of files currently completed compressed, total number of bytes compression progress, total number of files compression progress; the sub-compression download progress information includes: the start writing time of each file, the end time of each file and the number of written bytes of each file; the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the current completed compressed file number to the total file number.
Wherein, the overall compression progress information not only contains the individual progress of byte number and file number, but also contains the total byte number of the file and the overall compression weighted progress of the total compression item of the file. For example, if the first weight for the byte compression progress is 09 and the second weight for the file compression progress is 0.1, the overall compression weighted progress can be calculated using the following formula (2): overall compression weighted progress = total byte compression progress 0.9+ total file number compression progress 0.1 = current compression byte number/total byte number 0.9+ current compression file number/total file number 0.1 (2)
It follows that equation (2) is fixed to 0 for the empty directory-free scene when the weighting is calculated with respect to equation (1). For example, when it is determined that the current number of compressed words is 1500, the current number of compressed files is 3, the total number of bytes is 3000, and the total number of files is 6, the above values are substituted into formula (2), and the overall compression weighting progress is 0.5.
The method and the system carry out the compression downloading of the catalogue in the resource downloading mode, monitor the compression downloading process of the catalogue in real time in the compression downloading process, and feed back the compression downloading progress information obtained by monitoring to the client so as to monitor the compression downloading process of the request catalogue list in real time.
Example IV
Fig. 6 is a schematic structural diagram of a monitoring device for a directory compression downloading process according to an embodiment of the present application. As shown in fig. 6, the monitoring device for the directory compression download process includes: an aggregate compression request receiving module 410, a first compression download progress information obtaining module 420, a second compression download progress information obtaining module 430, and a monitoring module 440.
An aggregate compression request receiving module 410, configured to receive an aggregate compression request sent by a client, where the aggregate compression request includes a request directory list and a compression download mode;
the first compression download progress information obtaining module 420 is configured to, when determining that the compression download mode is a complete directory download mode, generate a file list to be compressed and an empty directory list according to the request directory list, and monitor a compression download progress of the file list to be compressed and the empty directory list to obtain compression download progress information in the complete directory download mode;
the second compression download progress information obtaining module 430 is configured to, when determining that the compression download mode is the resource download mode, generate a file list to be compressed according to the request directory list, and monitor a compression download progress of the file list to be compressed to obtain compression download progress information in the resource download mode;
The monitoring query request receiving module 440 is configured to, when receiving a progress monitoring query request sent by the client, send compressed download progress information matched with the compressed download mode to the client.
Optionally, the first compression downloading progress information obtaining module includes a list generating sub-module, configured to recursively traverse the request directory and a multi-level sub-directory of the request directory to obtain all nodes, where the nodes include a directory and a valid file;
determining an empty directory in the node, wherein the empty directory is a directory in the node which does not contain any child node;
and constructing a file list to be compressed according to all the effective files, and constructing an empty directory list according to the empty directory.
Optionally, the manifest generating submodule includes an empty catalog manifest generating unit, configured to obtain a preset arrangement rule;
and ordering the empty catalogues according to the arrangement rule to construct an empty catalogue list.
Optionally, the second compression downloading progress information obtaining module includes a file list to be compressed generating sub-module, configured to determine valid files in all nodes;
and constructing a file list to be compressed according to the directory effective file.
Optionally, the to-be-compressed file list generating sub-module is further configured to obtain attribute information of the effective file, where the attribute information includes a file identifier and a file size;
And constructing a file list to be compressed according to the attribute information of the effective file.
Optionally, the device further includes a first binding module, configured to create a compression instance and a monitoring object of the compression instance through the compression module by using the directory;
binding the output stream of the compression instance with the monitoring object, and communicating the monitoring object with the response output stream of the aggregation compression request.
Optionally, the first compression downloading progress information obtaining module includes a first compression downloading sub-module, configured to call a file resource corresponding to a file list to be compressed to a resource server, and perform compression downloading on the file resource and each empty directory in the empty directory list;
the first monitoring sub-module is used for monitoring the compression downloading process of each file resource and each empty directory in the empty directory list and obtaining compression progress information in a complete directory downloading mode.
Optionally, the first monitoring sub-module is configured to call a file resource corresponding to the file list to be compressed to the resource server, and compress and download the file resource;
and monitoring the compression downloading process of the file resource to obtain compression progress information in a resource downloading mode.
Optionally, the first compression download sub-module is configured to create a file compression class of each effective file in the file list to be compressed by running the compression instance, and call a file resource of each effective file to the resource server, where the file compression class includes attribute information of the file;
Compressing the file compression class and the file resource to obtain a file compression downloading result;
creating a catalog compression class of each empty catalog in the empty catalog list by running a compression example, wherein the catalog compression class contains attribute information of the empty catalog;
and compressing the catalog compression class to obtain an empty catalog compression downloading result.
Optionally, the second compression downloading progress information obtaining module includes a second compression downloading sub-module, configured to create a file compression class of each effective file in the file list to be compressed by running a compression instance, and call a file resource of each effective file to the resource server, where the file compression class includes attribute information of the file;
and compressing the file compression class and the file resource to obtain a file compression downloading result.
Optionally, the second compression download sub-module is further configured to traverse the file list to be compressed by running a compression instance;
when the target file which is not downloaded in a compression mode is contained in the file list to be compressed according to the traversing result, creating a file compression class for the target file, and storing attribute information of the target file into the file compression class;
and sending a resource reading request for the target file to a resource server, receiving the target file resource fed back by the resource server based on the resource reading request, and writing the target file resource into the compression instance in the form of byte stream.
Optionally, the device further includes a first refreshing module, configured to perform forced refreshing processing on an output buffer of the compression instance, so as to ensure that all the compression results downloaded by the non-empty directory are transmitted to the response output stream.
Optionally, the first compression download sub-module is further configured to traverse the empty directory list by running a compression instance;
when the target empty catalogue which is not downloaded in a compression mode is determined to be contained in the empty catalogue list according to the traversing result, creating a catalogue compression class for the target empty catalogue;
and saving the attribute information of the target empty directory into a directory compression class.
Optionally, the device further comprises a second refreshing module, configured to perform forced refreshing processing on the output buffer of the compression instance, so as to ensure that the results of the compression and downloading of the empty directory are all transmitted to the response output stream;
and closing the compression instance when the fact that all the empty directories in the empty directory list are compressed is determined to be completed.
Optionally, the first monitoring submodule is used for monitoring the output of the compression instance by the monitoring object in the monitoring module to obtain the number of bytes currently compressed and the final generation time;
and monitoring the operation of the compression instance through a monitoring module, and acquiring the integral compression downloading progress, integral compression weighting progress and sub-compression downloading progress information of each effective file and each empty directory aiming at the file list to be compressed and the empty directory list in a complete directory downloading mode.
Optionally, the overall compression downloading progress information includes: start writing time, compression completion time, current written byte number and last writing time, current completed compressed file number, current completed compressed directory number, total byte number compression progress and total file number compression progress;
the sub-compression download progress information includes: the start writing time of each file, the end time of each file, the number of bytes written in each file, the start writing time of each empty directory, the end time of each empty directory and the number of bytes written in each empty directory;
the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the sum of the current completed compressed file number and the current completed compressed directory number to the sum of the total file number and the total empty directory number.
Optionally, the second compression downloading progress information obtaining module includes a second monitoring sub-module, configured to monitor, by using a monitoring object in the monitoring module, an output of the compression instance to obtain a current number of bytes that have been compressed and a last generation time;
And monitoring the operation of the compression instance through a monitoring module, and acquiring the overall compression downloading progress, the overall compression weighting progress and the sub-compression downloading progress information of each effective file aiming at the file list to be compressed in a resource downloading mode.
Optionally, the overall compression downloading progress information includes: start writing time, compression completion time, number of bytes currently written and last writing time, number of files currently completed compressed, total number of bytes compression progress, total number of files compression progress;
the sub-compression download progress information includes: the start writing time of each file, the end time of each file and the number of written bytes of each file;
the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the current completed compressed file number to the total file number.
Optionally, the monitoring query request receiving module is used for extracting a request monitoring type from the progress monitoring query request;
and calling target compression downloading progress information corresponding to the request monitoring type, and sending the target compression downloading progress information to the client so as to display the target compression downloading progress information on the client.
It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element here may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
Example five
Fig. 7 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the monitoring method of the directory compression download process.
In some embodiments, the method of monitoring the directory compression download process may be implemented as a computer program tangibly embodied on a computer readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the above-described monitoring method of the directory compression download process may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the monitoring method of the directory compression download process in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
The embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements a method of monitoring a directory compression download process as provided by any of the embodiments of the present application.
Computer program product in the implementation, the computer program code for carrying out operations of the present application may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims (22)

1. The monitoring method for the directory compression downloading process is characterized by being applied to a file compression server and comprising the following steps:
receiving an aggregation compression request sent by a client, wherein the aggregation compression request comprises a request directory list and a compression downloading mode;
when the compression downloading mode is determined to be a complete catalog downloading mode, generating a file list to be compressed and an empty catalog list according to the request catalog list, and monitoring the compression downloading process of the file list to be compressed and the empty catalog list to obtain compression downloading progress information in the complete catalog downloading mode, wherein the complete catalog downloading mode indicates that an effective file in a catalog is downloaded and an empty catalog which does not contain the effective file is downloaded at the same time, and the effective file refers to a file which is contained in a non-empty catalog and can be effectively compressed and downloaded;
When the compression download mode is determined to be a resource download mode, generating a file list to be compressed according to the request directory list, and monitoring the compression download progress of the file list to be compressed to obtain compression download progress information in the resource download mode, wherein the resource download mode represents that only effective files in a directory are downloaded;
and when a progress monitoring query request sent by the client is received, the compressed downloading progress information matched with the compressed downloading mode is sent to the client.
2. The method of claim 1, wherein the file compression server includes a compression module and a monitoring module, and the receiving the aggregate compression request sent by the client further includes:
creating a compression instance and a monitoring object of the compression instance through the compression module;
binding the output stream of the compression instance with the monitoring object, and communicating the monitoring object with the response output stream of the aggregation compression request.
3. The method of claim 2, wherein generating a file list to be compressed and an empty directory list from the request directory list comprises:
Recursively traversing the request directory and the multi-level subdirectories of the request directory to obtain all nodes, wherein the nodes comprise directories and effective files;
determining an empty directory in the node, wherein the empty directory is a directory in the node which does not contain any child node;
and constructing the file list to be compressed according to the effective files in all the nodes, and constructing an empty directory list according to the empty directory.
4. A method according to claim 3, wherein said constructing an empty directory listing from said empty directory comprises:
acquiring a preset arrangement rule;
and ordering the empty catalogues according to the arrangement rule to construct the empty catalogue list.
5. A method according to claim 3, wherein said generating a list of files to be compressed from said request directory list comprises:
determining valid files in all nodes;
and constructing a file list to be compressed according to the effective file.
6. The method of claim 5, wherein constructing a list of files to be compressed from the active file comprises:
acquiring attribute information of an effective file, wherein the attribute information comprises a file identifier and a file size;
And constructing the file list to be compressed according to the attribute information of the effective file.
7. The method of claim 3, wherein monitoring the compression download progress of the file list to be compressed and the empty directory list to obtain the compression download progress information in the complete directory download mode comprises:
file resources corresponding to the file list to be compressed are called from a resource server, and the file resources and each empty directory in the empty directory list are compressed and downloaded;
and monitoring the compression downloading process of each file resource and each empty directory in the empty directory list to obtain compression progress information in a complete directory downloading mode.
8. The method of claim 5, wherein monitoring the compression download progress of the file list to be compressed to obtain the compression download progress information in the resource download mode comprises:
file resources corresponding to the file list to be compressed are called from a resource server, and the file resources are compressed and downloaded;
and monitoring the compression downloading process of the file resource to obtain compression progress information in a resource downloading mode.
9. The method of claim 7, wherein the retrieving the file resource corresponding to the file list to be compressed from the resource server and compressing and downloading each empty directory in the file resource and the empty directory list comprises:
creating a file compression class of each effective file in the file list to be compressed by running the compression example, and calling file resources of each effective file to the resource server, wherein the file compression class contains attribute information of the file;
compressing the file compression class and the file resource to obtain a file compression downloading result;
creating a catalog compression class of each empty catalog in the empty catalog list by running the compression example, wherein the catalog compression class contains attribute information of the empty catalog;
and compressing the catalog compression class to obtain an empty catalog compression downloading result.
10. The method of claim 8, wherein the retrieving the file resource corresponding to the file list to be compressed from the resource server and performing compression downloading on the file resource comprises:
creating a file compression class of each effective file in the file list to be compressed by running the compression example, and calling file resources of each effective file to the resource server, wherein the file compression class contains attribute information of the file;
And compressing the file compression class and the file resource to obtain a file compression downloading result.
11. The method according to claim 9 or 10, wherein creating a file compression class of each valid file in the file list to be compressed by running the compression instance, and calling file resources of each valid file to the resource server, comprises:
traversing the file list to be compressed by running the compression example;
when the target file which is not downloaded in a compression mode is contained in the file list to be compressed according to the traversing result, creating the file compression class for the target file, and storing attribute information of the target file into the file compression class;
and sending a resource reading request aiming at the target file to the resource server, receiving the target file resource fed back by the resource server based on the resource reading request, and writing the target file resource into the compression instance in a byte stream form.
12. The method according to claim 9 or 10, wherein after compressing the file compression class and the file resource to obtain a file compression download result, the method further comprises:
And carrying out forced refreshing treatment on the output buffer memory of the compression example to ensure that the file compression downloading result is completely transmitted to the response output stream.
13. The method of claim 9, wherein creating a directory compression class for each empty directory in the empty directory list by running the compression instance comprises:
traversing the empty directory listing by running the compression instance;
when the target empty catalogue which is not downloaded in a compression mode is determined to be contained in the empty catalogue list according to the traversing result, creating the catalogue compression class for the target empty catalogue;
and storing the attribute information of the target empty directory into the directory compression class.
14. The method of claim 13, wherein after compressing the directory compression class to obtain an empty directory compression download result, further comprising:
the output buffer memory of the compression example is subjected to forced refreshing directory processing so as to ensure that the whole transmission of the empty directory compression downloading result to the response output stream is ensured;
and closing the compression example when the fact that all the empty directories in the empty directory list are compressed is determined to be completed.
15. The method of claim 7, wherein monitoring the compression download progress of each of the file resources and each of the empty directories in the empty directory list to obtain compression progress information in a complete directory download mode comprises:
The catalogue monitors the output of the compression instance through the monitoring object in the monitoring module to obtain the number of bytes currently compressed and the final generation time;
and monitoring the operation of the compression example through the monitoring module to obtain the integral compression downloading progress, integral compression weighting progress and sub-compression downloading progress information of each effective file and each empty directory aiming at the file list to be compressed and the empty directory list in a complete directory downloading mode.
16. The method of claim 15, wherein the overall compression download schedule comprises: start writing time, compression completion time, current written byte number and last writing time, current completed compressed file number, current completed compressed directory number, total byte number compression progress and total file number compression progress;
the sub-compression download progress information includes: the start writing time of each file, the end time of each file, the number of bytes written in each file, the start writing time of each empty directory, the end time of each empty directory and the number of bytes written in each empty directory;
the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the sum of the current completed compressed file number and the current completed compressed directory number to the sum of the total file number and the total empty directory number.
17. The method of claim 7, wherein monitoring the compression download process of the file resource to obtain compression progress information in a resource download mode comprises:
monitoring the output of the compression instance through the monitoring object in the monitoring module to obtain the number of bytes currently compressed and the final generation time;
and monitoring the operation of the compression instance through the monitoring module, and acquiring the overall compression downloading progress, the overall compression weighting progress and the sub-compression downloading progress information of each effective file aiming at the file list to be compressed in a resource downloading mode.
18. The method of claim 17, wherein the overall compression download schedule comprises: start writing time, compression completion time, number of bytes currently written and last writing time, number of files currently completed compressed, total number of bytes compression progress, total number of files compression progress;
the sub-compression download progress information includes: the start writing time of each file, the end time of each file and the number of written bytes of each file;
the overall compression weighting progress is the sum of the total byte compression progress multiplied by the sum of the first weight and the total file number compression progress multiplied by the second weight, wherein the total byte number compression progress is the ratio of the current written byte number to the total byte number, and the total file number compression progress is the ratio of the current completed compressed file number to the total file number.
19. The method of claim 1, wherein the transmitting compressed download progress information matched to the compressed download pattern to the client comprises:
extracting a request monitoring type from the progress monitoring query request;
and retrieving target compression downloading progress information corresponding to the request monitoring type, and sending the target compression downloading progress information to a client so as to display the target compression downloading progress information on the client.
20. A monitoring device for a directory compression download process, comprising:
the aggregation compression request receiving module is used for receiving an aggregation compression request sent by the client, wherein the aggregation compression request comprises a request catalog list and a compression downloading mode;
the first compression downloading progress information acquisition module is used for generating a file list to be compressed and an empty directory list according to the request directory list when the compression downloading mode is determined to be a complete directory downloading mode, and monitoring the compression downloading progress of the file list to be compressed and the empty directory list to acquire compression downloading progress information in the complete directory downloading mode, wherein the complete directory downloading mode indicates that an effective file in a directory is downloaded and an empty directory which does not contain the effective file is downloaded at the same time, and the effective file refers to a file which is contained in a non-empty directory and can be effectively compressed and downloaded;
The second compression downloading progress information acquisition module is used for generating a file list to be compressed according to the request directory list when the compression downloading mode is determined to be a resource downloading mode, and monitoring the compression downloading progress of the file list to be compressed to acquire the compression downloading progress information in the resource downloading mode, wherein the resource downloading mode represents that only effective files in a directory are downloaded;
and the monitoring query request receiving module is used for sending the compressed downloading progress information matched with the compressed downloading mode to the client when receiving the progress monitoring query request sent by the client.
21. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-19.
22. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-19.
CN202310876743.4A 2023-07-18 2023-07-18 Method, device and program product for monitoring directory compression downloading process Active CN116610643B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310876743.4A CN116610643B (en) 2023-07-18 2023-07-18 Method, device and program product for monitoring directory compression downloading process

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310876743.4A CN116610643B (en) 2023-07-18 2023-07-18 Method, device and program product for monitoring directory compression downloading process

Publications (2)

Publication Number Publication Date
CN116610643A CN116610643A (en) 2023-08-18
CN116610643B true CN116610643B (en) 2023-10-17

Family

ID=87675041

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310876743.4A Active CN116610643B (en) 2023-07-18 2023-07-18 Method, device and program product for monitoring directory compression downloading process

Country Status (1)

Country Link
CN (1) CN116610643B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6195678B1 (en) * 1996-09-03 2001-02-27 Fujitsu Limited Remote resource management system for automatically downloading required files from application server depending on contents of selected files on requesting computer
WO2003073195A2 (en) * 2001-12-12 2003-09-04 Valve Corporation Method and system for effectively communicating file properties and directory structures in a distributed file system
CN103631604A (en) * 2013-12-17 2014-03-12 华中师范大学 Standardized digital learning resource management method and system based on CELTS (Chinese E-learning technology standard)-41 specification
CN109063468A (en) * 2018-06-14 2018-12-21 珠海金山网络游戏科技有限公司 A kind of installation kit integrity verification and the method for reparation
CN110210761A (en) * 2019-06-04 2019-09-06 北京中标新亚节能工程股份有限公司 Structure adjusting is matched and information management system
CN110650313A (en) * 2019-09-12 2020-01-03 视联动力信息技术股份有限公司 Catalog display method and device for monitoring resources and storage medium
CN110913017A (en) * 2019-12-19 2020-03-24 深圳市原创科技有限公司 File compression transmission method based on cloud desktop
CN112714199A (en) * 2021-03-29 2021-04-27 北京拓课网络科技有限公司 Courseware loading method and system and electronic equipment
CN114448972A (en) * 2021-12-30 2022-05-06 苏州浪潮智能科技有限公司 Distributed storage log compression downloading method, system, terminal and storage medium
EP4161022A1 (en) * 2021-09-29 2023-04-05 Nokia Technologies Oy Method to manage file downloads

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11064010B2 (en) * 2018-03-30 2021-07-13 Microsoft Technology Licensing, Llc Download progress information for composite files

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6195678B1 (en) * 1996-09-03 2001-02-27 Fujitsu Limited Remote resource management system for automatically downloading required files from application server depending on contents of selected files on requesting computer
WO2003073195A2 (en) * 2001-12-12 2003-09-04 Valve Corporation Method and system for effectively communicating file properties and directory structures in a distributed file system
CN103631604A (en) * 2013-12-17 2014-03-12 华中师范大学 Standardized digital learning resource management method and system based on CELTS (Chinese E-learning technology standard)-41 specification
CN109063468A (en) * 2018-06-14 2018-12-21 珠海金山网络游戏科技有限公司 A kind of installation kit integrity verification and the method for reparation
CN110210761A (en) * 2019-06-04 2019-09-06 北京中标新亚节能工程股份有限公司 Structure adjusting is matched and information management system
CN110650313A (en) * 2019-09-12 2020-01-03 视联动力信息技术股份有限公司 Catalog display method and device for monitoring resources and storage medium
CN110913017A (en) * 2019-12-19 2020-03-24 深圳市原创科技有限公司 File compression transmission method based on cloud desktop
CN112714199A (en) * 2021-03-29 2021-04-27 北京拓课网络科技有限公司 Courseware loading method and system and electronic equipment
EP4161022A1 (en) * 2021-09-29 2023-04-05 Nokia Technologies Oy Method to manage file downloads
CN114448972A (en) * 2021-12-30 2022-05-06 苏州浪潮智能科技有限公司 Distributed storage log compression downloading method, system, terminal and storage medium

Also Published As

Publication number Publication date
CN116610643A (en) 2023-08-18

Similar Documents

Publication Publication Date Title
CN113568884B (en) File management method and device, electronic equipment and storage medium
CN103095819A (en) Data information pushing method and data information pushing system
CN107689976B (en) File transmission method and device
JP5986602B2 (en) Method, apparatus, terminal and server for synchronizing an exact copy of a terminal data set
CN113961510B (en) File processing method, device, equipment and storage medium
CN112256772A (en) Data service method, device and readable storage medium
CN113411404A (en) File downloading method, device, server and storage medium
CN110377419A (en) A kind of server calls method, apparatus and electronic equipment
CN114448972A (en) Distributed storage log compression downloading method, system, terminal and storage medium
CN116610643B (en) Method, device and program product for monitoring directory compression downloading process
CN112926002A (en) Service request processing method and device
CN115934076B (en) Cross-platform client micro-service providing device and method and electronic equipment
CN111294377B (en) Dependency network request sending method, terminal device and storage medium
CN109218371B (en) Method and equipment for calling data
CN110347380A (en) Front and back end development approach and equipment
CN114567571A (en) Performance test method and device, electronic equipment and computer readable storage medium
CN114238264A (en) Data processing method, data processing device, computer equipment and storage medium
CN108805741B (en) Fusion method, device and system of power quality data
CN115129708A (en) Data processing method, data processing device, storage medium and electronic equipment
CN111611077A (en) Task parameter processing method, terminal and storage medium
JP6734303B2 (en) Information processing method and device
CN108399223A (en) A kind of data capture method, device and electronic equipment
CN114817166B (en) Method, device and medium for canceling file operation
CN117454036A (en) Method, device, computer equipment and storage medium for reporting data collection
CN114860469A (en) Data acquisition method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant