CN113517985B - File data processing method and device, electronic equipment and computer readable medium - Google Patents

File data processing method and device, electronic equipment and computer readable medium Download PDF

Info

Publication number
CN113517985B
CN113517985B CN202110777032.2A CN202110777032A CN113517985B CN 113517985 B CN113517985 B CN 113517985B CN 202110777032 A CN202110777032 A CN 202110777032A CN 113517985 B CN113517985 B CN 113517985B
Authority
CN
China
Prior art keywords
file
rule
request
client
uploading
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110777032.2A
Other languages
Chinese (zh)
Other versions
CN113517985A (en
Inventor
覃朗
陈海川
廖谦
白晓磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202110777032.2A priority Critical patent/CN113517985B/en
Publication of CN113517985A publication Critical patent/CN113517985A/en
Application granted granted Critical
Publication of CN113517985B publication Critical patent/CN113517985B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/321—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority
    • H04L9/3213—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority using tickets or tokens, e.g. Kerberos
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/0807—Network architectures or network communication protocols for network security for authentication of entities using tickets, e.g. Kerberos
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]

Abstract

The invention discloses a file data processing method, a file data processing device, electronic equipment and a computer readable medium, and relates to the technical field of big data.A specific implementation mode comprises the steps of receiving an uploading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded, and returning the website rule configuration information; and receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request to upload the newly added file, and returning uploading progress and address information of the newly added file. Therefore, the method and the device can solve the problems of low file management efficiency and poor safety in the prior art.

Description

File data processing method and device, electronic equipment and computer readable medium
Technical Field
The invention relates to the technical field of big data, and provides a file data processing method and device, electronic equipment and a computer readable medium.
Background
With the development of business, the requirements for uploading and archiving files such as videos, voices and pictures are increasing, and meanwhile, the banking business requires high real-time performance, so that the file uploading cannot influence important transactions. And part of intra-row service departments need the uploading requirement of uploading 10G to 20G super-large files, namely managing the super-large files. Cross-domain support is needed for a browser request, and APP and client access service are supported at the same time.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
currently, file uploading in the industry is generally performed by HTTP, FTP protocol and file segment uploading based on HTTP protocol. However, file uploads based on HTTP, FTP protocols are unstable, fail requiring retransmission, bandwidth sensitive, and file size is limited by the server side. In addition, at present, no tool for increasing the functions of uploading and downloading super-large files for controlling different business rules and supporting breakpoint continuous transmission, speed limitation and data safety exists in a bank client. And the aspect of the server side does not have a component integrating the functions of security access, large file storage and shared files, flow control and cross-domain function.
Disclosure of Invention
In view of this, embodiments of the present invention provide a file data processing method and apparatus, an electronic device, and a computer readable medium, which can solve the problems of low file management efficiency and poor security in the prior art.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, a file data processing method is provided, which is applied to a server, and includes receiving an upload rule request sent by a client, invoking a rule engine to generate website rule configuration information based on client information in the upload rule request and attribute information of a file to be uploaded, and returning the website rule configuration information; and receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request to upload the newly added file, and returning uploading progress and address information of the newly added file.
Optionally, the method comprises the following steps:
receiving a Token request sent by a client, verifying an authorization key, and calling the number of client website requests recorded in a cache after the authorization key passes verification;
judging whether the request quantity is larger than a preset concurrency threshold value or not, if so, returning a current-limiting identifier and a generated Token, receiving an uploading rule request sent by a client, verifying the Token in the uploading rule request, calling a rule engine according to the current-limiting identifier after passing verification, and generating website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded; and if not, returning the generated Token, receiving an uploading rule request sent by the client, verifying the Token in the uploading rule request, and calling a rule engine to generate website rule configuration information based on the client information in the uploading rule request and the attribute information of the file to be uploaded after the Token passes verification.
Optionally, the uploading of the newly added file is executed, including:
and calling a preset current limiting algorithm, and triggering an application current limiting program uploaded to the newly added file according to the network point rule configuration information.
Optionally, triggering an application current limiting program for uploading the newly added file according to the network node rule configuration information includes:
judging whether the file uploading request meets an access rule in the node rule configuration information;
if so, carrying out multi-thread file transmission according to bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and mesh point rule configuration information;
and if not, rejecting the file uploading request.
Optionally, after performing multi-thread file transmission according to the bandwidth limitation, the method includes:
and monitoring file transmission failure and triggering a delayed retransmission program.
Optionally, before receiving the upload rule request sent by the client, the method includes:
and acquiring the target attribute of the client, determining that the file uploading function is allowed to be used, and then returning the uploading file master switch parameter to the client so as to start the file uploading plug-in of the client.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a file data processing method, applied to a client, including: acquiring attribute information and client information of a file to be uploaded, generating an uploading rule request and sending the uploading rule request to a server; receiving network point rule configuration information to perform fragmentation processing on a file to be uploaded, generating a file uploading request and sending the file uploading request to a server; and receiving and storing the uploading progress and the address information of the newly added file.
Optionally, the obtaining attribute information of the file to be uploaded and the client information, and generating an upload rule request includes:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token;
and acquiring attribute information and client information of the file to be uploaded, and generating an uploading rule request based on the current limiting identifier and Token.
Optionally, after receiving and storing the uploading progress and the address information of the newly added file, the method includes:
and receiving a delayed retransmission instruction, positioning the fragmented file, and generating and sending a file uploading request according to the node rule configuration information.
Optionally, before generating the upload rule request and sending the request to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of an uploaded file;
and starting a file uploading plug-in according to the uploading file main switch parameter.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a file data processing method, applied to a server, including: receiving a downloading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information and attribute information of a file to be downloaded in the downloading rule request, and returning the website rule configuration information; and receiving a file downloading request sent by the client according to the website rule configuration information, and inquiring file address information in the file downloading request to execute file downloading and return.
Optionally, the method comprises the following steps:
receiving a Token request sent by a client, verifying an authorization key, and calling the number of client website requests recorded in a cache after the authorization key passes verification;
judging whether the number of the requests is greater than a preset concurrency threshold value, if so, returning a current-limiting identifier and a generated Token, receiving a downloading rule request sent by a client, verifying the Token in the downloading rule request, calling a rule engine according to the current-limiting identifier after the verification is passed, and generating website rule configuration information based on client information in the downloading rule request and attribute information of a file to be downloaded; and if not, returning the generated Token, receiving a download rule request sent by the client, verifying the Token in the download rule request, and calling a rule engine to generate website rule configuration information based on the client information in the download rule request and the attribute information of the file to be downloaded after the Token passes verification.
Optionally, performing a file download, comprising:
and calling a preset current limiting algorithm, and triggering an application current limiting program for downloading the file according to the network point rule configuration information.
Optionally, triggering an application current limiting program for downloading a file according to the website rule configuration information includes:
judging whether the file downloading request meets an access rule in the node rule configuration information or not;
if yes, carrying out multi-thread file transmission according to the bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and the mesh point rule configuration information;
if not, rejecting the file downloading request.
Optionally, before receiving the request for the download rule sent by the client, the method includes:
and acquiring the target attribute of the client, determining that the file downloading function is allowed to be used, and then returning the main switch parameter of the downloaded file to the client so as to start the file downloading plug-in of the client.
In order to achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a file data processing method, applied to a client, including: acquiring attribute information and client information of a file to be downloaded, generating a downloading rule request and sending the downloading rule request to a server; and receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, sending the file downloading request to the server, and further receiving a file downloading result returned by the server.
Optionally, obtaining attribute information of the file to be downloaded and client information, and generating a download rule request, including:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token;
and acquiring attribute information and client information of the file to be downloaded, and generating a downloading rule request based on the current limiting identifier and Token.
Optionally, before generating the request for the download rule and sending the request to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of a downloaded file;
and starting the file downloading plug-in according to the download file master switch parameter.
In addition, the invention also provides a file data processing device which is applied to a server and comprises a configuration module, a rule engine and a data processing module, wherein the configuration module is used for receiving the uploading rule request sent by the client, calling the rule engine to generate website rule configuration information based on the client information in the uploading rule request and the attribute information of the file to be uploaded, and returning the website rule configuration information; or the rule engine is called to generate website rule configuration information and return the website rule configuration information based on the client information in the downloading rule request and the attribute information of the file to be downloaded;
the uploading and downloading service module is used for receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request so as to upload a newly added file, and returning an uploading progress and address information of the newly added file; or the server is used for receiving a file downloading request sent by the client according to the website rule configuration information, inquiring file address information in the file downloading request, and executing file downloading and returning.
Optionally, comprising:
the security module is used for receiving a Token request sent by the client and verifying the authorization key;
the network node current limiting module is used for calling the client network node request quantity recorded in the cache after the verification is passed; judging whether the number of the requests is larger than a preset concurrency threshold value, if so, returning a current-limiting identifier and the generated Token; and if not, returning the generated Token.
Optionally, comprising:
the application current limiting module is used for calling a preset current limiting algorithm and triggering an application current limiting program uploaded to the newly added file according to the network point rule configuration information; or calling a preset current limiting algorithm, and triggering an application current limiting program for downloading the file according to the network point rule configuration information.
Optionally, the triggering, by the application current limiting module, the application current limiting program uploaded to the newly added file according to the node rule configuration information or the application current limiting program downloaded to the file according to the node rule configuration information includes:
judging whether the file uploading request or the file downloading request meets an access rule in the website rule configuration information;
if yes, carrying out multi-thread file transmission according to the bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and the mesh point rule configuration information;
and if not, rejecting the file uploading request or rejecting the file downloading request.
Optionally, after the applying the throttling module performs the multi-thread file transmission according to the bandwidth limitation, the applying the throttling module includes:
and monitoring file transmission failure and triggering a delayed retransmission program.
Optionally, before the configuration module receives the upload rule request sent by the client or receives the download rule request sent by the client, the method includes:
and acquiring the target attribute of the client, determining the function of allowing to use the file uploading or downloading, and then returning the total file uploading switch parameter or the total file downloading switch parameter to the client so as to start the file uploading plug-in or the file downloading plug-in of the client.
In addition, the invention also provides a file data processing device, which is applied to a client and comprises the following components: the uploading module is used for acquiring attribute information and client information of a file to be uploaded, generating an uploading rule request and sending the uploading rule request to the server; receiving network point rule configuration information to perform fragmentation processing on a file to be uploaded, generating a file uploading request and sending the file uploading request to a server; receiving and storing the uploading progress and the address information of the newly added file;
the downloading module is used for acquiring attribute information and client information of a file to be downloaded, generating a downloading rule request and sending the downloading rule request to the server; and receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, sending the file downloading request to the server, and further receiving a file downloading result returned by the server.
Optionally, the upload module is further configured to:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; and acquiring attribute information and client information of the file to be uploaded, and generating an uploading rule request based on the current limiting identifier and Token.
Optionally, after the uploading module receives and stores the uploading progress and the address information of the newly added file, the method includes:
and receiving a delayed retransmission instruction, positioning the fragmented file, and further generating a file uploading request according to the node rule configuration information and sending the file uploading request.
Optionally, before the uploading module generates the uploading rule request and sends the uploading rule request to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of an uploaded file;
and starting a file uploading plug-in according to the uploading file main switch parameter.
Optionally, the downloading module is further configured to:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; and acquiring attribute information and client information of the file to be downloaded, and generating a downloading rule request based on the current limiting identifier and Token.
Optionally, before the downloading module generates the request of the downloading rule and sends the request to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of a downloaded file;
and starting the file downloading plug-in according to the download file master switch parameter.
Optionally, the method further comprises:
the management module is used for storing the rule configuration information of the network nodes;
and the UI interface module is used for providing an interface for uploading or downloading files.
One embodiment of the above invention has the following advantages or benefits: the invention provides a whole set of ultra-large file transmission and management of the banking and financial industry, which can realize a segmented uploading technology based on an HTTP protocol, carry out file verification, flow control and safe access control on file uploading through customizing different uploading and downloading rules and combining joint control of a server and a client, and meet the requirements of different business scenes, namely, the multi-configuration dimensionality is suitable for special business scenes of banks. And flexible uploading and downloading speed configuration is supported, and the method is more suitable for scenes with high bandwidth requirements. The control rule configuration and the uploading and downloading speed control flexibly adapt to various banking rule scenes. And the data routing is carried out according to the service scene or the service place to carry out flow distribution. Moreover, the invention realizes the safe access of the interface, the successful authorization for uploading and downloading the file, the invalidation of the expired TOKEN and the embezzlement of the crawler. And the retransmission fails, so that the data can be continuously uploaded every other day and every other week even if the network is interrupted, and the use efficiency of the bandwidth is effectively improved. In addition, the method supports the transmission of the super-large files and meets the file transmission requirements of users at the server and the client.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic view of a main flow of a document data processing method according to a first embodiment of the present invention;
FIG. 2 is a schematic view of a main flow of a document data processing method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of the main flow of launching a file transfer plug-in according to an embodiment of the invention;
FIG. 4 is a schematic diagram of a main flow of an application current limiting procedure according to an embodiment of the present invention;
FIG. 5 is a diagram showing a main flow of a document data processing method according to a third embodiment of the present invention;
FIG. 6 is a schematic view of a main flow of a document data processing method according to a fourth embodiment of the present invention;
FIG. 7 is a schematic diagram of the main blocks of a document data processing apparatus according to a first embodiment of the present invention;
FIG. 8 is a schematic diagram of the main blocks of a document data processing apparatus according to a second embodiment of the present invention;
FIG. 9 is an architectural diagram of a client according to an embodiment of the invention;
FIG. 10 is a schematic diagram of main blocks of a document data processing apparatus according to a third embodiment of the present invention;
FIG. 11 is a block diagram of a server according to an embodiment of the present invention;
FIG. 12 is a sequence diagram of a main flow of a file data processing method according to an embodiment of the present invention;
FIG. 13 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 14 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a file data processing method according to a first embodiment of the present invention, as shown in fig. 1, the file data processing method is applied to an upload file of a server, and the file data processing method includes:
and step S101, receiving an uploading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded, and returning.
Step S102, receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request to upload the newly added file, and returning uploading progress and address information of the newly added file.
Fig. 2 is a schematic diagram of a main flow of a file data processing method according to a second embodiment of the present invention, as shown in fig. 2, the file data processing method is applied to a download file of a server, and the file data processing method includes:
step S201, receiving a downloading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information and attribute information of a file to be downloaded in the downloading rule request, and returning.
Step S202, receiving a file downloading request sent by the client according to the website rule configuration information, and inquiring file address information in the file downloading request to execute file downloading and return.
As an embodiment, when step S101 or step S201 is executed, a weak access control mechanism may be performed based on an upload and download rule, and the server obtains corresponding website rule configuration information from preset rule information including IP, a file menu, a usage scenario, a rule switch, a service component switch, an upload allowed time period, a file type, a speed limit configuration, and the like according to client information (e.g., organization code, IP, and the like) and file attribute information to be uploaded or file attribute information to be downloaded (e.g., a file menu, a service component code, and the like (the service component code is a service component ID defined according to different service types).
In some embodiments, the client requests TOKEN each time a file is uploaded or downloaded, and after the authorization key passes the verification, the amount of requests of the network node in each time interval is recorded by using a cache based on the current limiting configuration of the network node granularity. When the request quantity of the network point reaches the concurrency quantity configured by the rule, the server side returns the current limit or forbids uploading or downloading the identifier to the client side, and the client side takes corresponding measures according to the identifier. The specific implementation process comprises the following steps:
and receiving a Token request sent by the client, verifying the authorization key, and calling the number of client website requests recorded in the cache after the authorization key passes verification. And then, judging whether the number of the requests is greater than a preset concurrency threshold, if so, returning a current-limiting identifier and generating a Token, receiving an uploading rule request sent by a client, verifying the Token in the uploading rule request, and calling a rule engine according to the current-limiting identifier after the verification is passed so as to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded. And if not, returning the generated Token, receiving an uploading rule request sent by the client, verifying the Token in the uploading rule request, and calling a rule engine to generate website rule configuration information based on the client information in the uploading rule request and the attribute information of the file to be uploaded after the Token passes verification.
Or receiving a Token request sent by the client, verifying the authorization secret key, and calling the number of client website requests recorded in the cache after the authorization secret key passes verification. And then, judging whether the number of the requests is greater than a preset concurrency threshold value, if so, returning a current-limiting identifier and generating a Token, receiving a downloading rule request sent by a client, verifying the Token in the downloading rule request, calling a rule engine according to the current-limiting identifier after the verification is passed, and generating website rule configuration information based on client information to-be-downloaded file attribute information in the downloading rule request. And if not, returning the generated Token, receiving a download rule request sent by the client, verifying the Token in the download rule request, and calling a rule engine to generate website rule configuration information based on the client information attribute information to be downloaded in the download rule request after the Token passes verification.
That is to say, the present invention adopts a strong access control mechanism Token, preferably adopts a TOTP algorithm (english time-based one-time password), and can generate a Token with limited time efficiency based on three factors, namely, a service component code, an authorization key, and a time factor, and when the Token exceeds the usage time, the Token will fail to be verified.
And if the network node request quantity is greater than a preset concurrency threshold and less than a preset threshold, returning a current limiting identifier, and performing current limiting processing when an uploading rule request or a downloading rule request of the client is received. If the number of the website requests is larger than or equal to a preset threshold value, returning an uploading or downloading forbidding identifier, and not processing the request when an uploading rule request or a downloading rule request of the client is received.
In still other embodiments, before receiving an upload rule request sent by a client or before receiving a download rule request sent by the client, the target attribute of the client may be obtained, the file upload or download function is determined to be allowed to be used, and then the upload file master switch parameter or the download file master switch parameter is returned to the client to start a file upload plug-in or download file plug-in of the client. In a specific implementation process, as shown in fig. 3, after the client logs in, the client sends target attributes such as the current client ip, the first-level line number, the mechanism number, the MAC + machine name, and the like to the server, and the server can determine whether the client allows the file uploading function according to the target attributes, and if so, returns a total file uploading switch parameter to the client, and otherwise, returns a message that the file uploading function is not allowed to be used. In addition, the client can start the file uploading plug-in according to the uploading file general switch parameter, if the file uploading plug-in fails to be started, the file uploading local plug-in is not started, and the uploading method is called by JavaScript (js for short) codes to report errors.
Certainly, after the client logs in, the client sends target attributes such as the current client ip, the first-level line number, the mechanism number, the MAC + machine name and the like to the server, and the server can also judge whether the client allows the function of downloading the file according to the target attributes, if so, the server returns the total switch parameters of the downloaded file to the client, and if not, returns a message that the function of downloading the file is not allowed to be used. In addition, the client can start the file downloading plug-in according to the main switch parameter of the downloading file, if the starting fails, the file downloading local plug-in is not started, and the uploading method is called by the JavaScript code to report errors.
As another embodiment, a preset current limiting algorithm may be called in the process of executing step S102 or step S202, and an application current limiting program uploaded to a newly added file or an application current limiting program downloaded to a file may be triggered according to the node rule configuration information. Preferably, the token bucket algorithm is called to trigger the application current limiting program uploaded to the newly added file to trigger the application current limiting program downloaded to the file according to the network point rule configuration information. The token bucket algorithm is adopted to support three-level speed reduction control, one level is normal speed uploading and downloading, the second level is normal speed uploading and downloading and carries a speed reduction warning mark, and the third level directly refuses access. And dynamic loading speed limit configuration is supported, and when the access exceeds the capacity of the server, the configuration can be adjusted, so that degradation is realized without service.
In addition, it should be noted that, triggering the application current-limiting program uploaded to the newly added file or triggering the application current-limiting program downloaded to the file according to the website rule configuration information, as shown in fig. 4, includes:
step S401, a task is called and started.
At least one file fragment in the file uploading or file downloading process is used as one task, that is, a plurality of tasks can be provided.
Step S402, judging whether the file uploading request or the file downloading request meets the access rule in the website rule configuration information, if so, performing step S403, and if not, performing step S409.
Step S403, determining whether the bandwidth limitation is satisfied, if yes, performing step S404, and if not, performing step S405.
In step S404, the multithread file transfer is performed according to the bandwidth limitation, and step S406 is executed.
Wherein the transmission speed is determined based on a current limiting algorithm and mesh point rule configuration information.
Step S405, the task is set in a waiting state, multi-thread file transmission is carried out after the query flow meets the bandwidth limitation, and step S406 is executed.
Step S406, determining whether the transmission is successful, if so, performing step S407, otherwise, performing step S408.
Step S407, return the transmission result to the client.
Step S408, triggering a delayed retransmission procedure.
Step S409, rejects the file upload request or the file download request.
Fig. 5 is a schematic diagram of a main flow of a file data processing method according to a third embodiment of the present invention, as shown in fig. 5, the file data processing method is applied to an uploaded file of a client, and the file data processing method includes:
step S501, obtaining attribute information of a file to be uploaded and client information, generating an uploading rule request and sending the uploading rule request to a server.
Step S502, receiving the rule configuration information of the website to perform fragment processing on the file to be uploaded, generating a file uploading request and sending the file uploading request to the server.
And step S503, receiving and storing the uploading progress and the address information of the newly added file.
Fig. 6 is a schematic diagram of a main flow of a file data processing method according to a fourth embodiment of the present invention, as shown in fig. 6, the file data processing method is applied to a downloaded file of a server, and the file data processing method includes:
step S601, obtaining attribute information and client information of a file to be downloaded, generating a downloading rule request and sending the downloading rule request to a server.
Step S602, receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, and sending the file downloading request to the server.
Step S603, receiving a file download result returned by the server.
In an embodiment, when performing step S501 or step S601, the client sends a Token request to the server according to a preset authorization key, so as to receive the current limit identifier and the generated Token. And then, acquiring attribute information of the file to be uploaded or the file to be downloaded and client information, and further generating an uploading rule request or a downloading rule request based on the current limiting identifier and Token.
As another embodiment, before the upload rule request is generated in step S501 and sent to the server or the download rule request is generated in step S601 and sent to the server, the client obtains the client target attribute and sends the client target attribute to the server, so as to receive the total switch parameter of the upload file or the total switch parameter of the download file. And then, starting a file uploading plug-in or a file downloading plug-in according to the uploading file total switching parameter or the downloading file total switching parameter.
Fig. 7 is a schematic diagram of main modules of a file data processing apparatus according to a first embodiment of the present invention, as shown in fig. 7, applied to a server, and the file data processing apparatus includes a configuration module 701 and an upload and download service module 702. The configuration module 701 receives an upload rule request sent by a client, and invokes a rule engine to generate website rule configuration information based on client information in the upload rule request and attribute information of a file to be uploaded, and returns the website rule configuration information. Or the rule engine is called to generate website rule configuration information and return the website rule configuration information based on the client information and the attribute information of the file to be downloaded in the downloading rule request.
The upload and download service module 702 receives a file upload request sent by the client according to the website rule configuration information, queries metadata in the file upload request to upload a new file, and returns an upload progress and address information of the new file. Or the server is used for receiving a file downloading request sent by the client according to the website rule configuration information, inquiring file address information in the file downloading request, and executing file downloading and returning.
In some embodiments, the configuring module 701, before receiving the upload rule request sent by the client or before receiving the download rule request sent by the client, includes:
and acquiring the target attribute of the client, determining that the file uploading function or downloading is allowed to be used, and then returning the uploading file master switch parameter or the downloading file master switch parameter to the client so as to start the file uploading plug-in or the file downloading plug-in of the client.
As another embodiment, as shown in fig. 8, the file data processing apparatus may further include a configuration module 701, an upload and download service module 702, a security module 703, a network point current limiting module 704, and an application current limiting module 705. The security module 703 receives a Token request sent by the client, and verifies the authorization key. The node current limiting module 704 calls the number of client node requests recorded in the cache after passing the verification; judging whether the number of the requests is larger than a preset concurrency threshold value, if so, returning a current-limiting identifier and a generated Token, receiving an uploading rule request or a downloading rule request sent by a client by the configuration module 701, verifying the Token in the uploading rule request, and calling a rule engine according to the current-limiting identifier after the verification is passed so as to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded. Or generating the rule configuration information of the network points based on the client information in the uploading rule request and the attribute information of the file to be downloaded.
If not, returning the generated Token, the configuration module 701 receives an upload rule request sent by the client, invokes a rule engine to generate website rule configuration information based on the client information in the upload rule request, and returns the website rule configuration information. Or the configuration module 701 receives a download rule request sent by the client, and invokes the rule engine to generate website rule configuration information based on the client information in the download rule request and return the website rule configuration information.
The upload and download service module 702 receives a file upload request sent by the client according to the website rule configuration information, queries metadata in the file upload request to upload a new file, and returns an upload progress and address information of the new file. Or the uploading and downloading service module 702 receives a file downloading request sent by the client according to the website rule configuration information, and inquires file address information in the file downloading request to execute file downloading and return.
The application current limiting module 705 calls a preset current limiting algorithm and triggers an application current limiting program uploaded to the newly added file according to the network point rule configuration information. Or calling a preset current limiting algorithm, and triggering an application current limiting program for downloading the file according to the network point rule configuration information.
In some embodiments, the application throttling module 705 triggers an application throttling program uploaded to the newly added file according to the node rule configuration information or triggers an application throttling program downloaded to the file according to the node rule configuration information, including:
judging whether the file uploading request or the file downloading request meets an access rule in the website rule configuration information; if yes, carrying out multi-thread file transmission according to the bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and the mesh point rule configuration information; and if not, rejecting the file uploading request or rejecting the file downloading request.
In some embodiments, after the application throttling module 705 performs a multi-threaded file transfer according to a bandwidth limitation, the method includes: and monitoring file transmission failure and triggering a delayed retransmission program.
It should be noted that the document data processing method and the document data processing apparatus according to the present invention have corresponding relationships in the specific implementation contents, and therefore, the description of the repeated contents is omitted.
As still other embodiments, as shown in fig. 9, the architecture of the file upload and download process server includes: the device comprises a storage module, a cache, an NAS (network attached storage), a network point current limiting module, a configuration module, a safety module, an uploading and downloading service module and an application current limiting module. The storage module, the cache and the NAS can be used for storing various data of the server side. The network node flow limiting module controls flow according to different client network nodes, and the configuration module controls rules based on rule information in IP, file menu, use scene, rule switch, service component switch, uploading allowed time period, file type, speed limit configuration and the like, namely controls network node rule configuration information. And the security module adopts a TOTP algorithm based on the strong access control mechanism Token, generates a Token with limited timeliness through the three elements of the service component code, the authorization key and the time factor, and sends the Token to the client for verifying the uploading rule request or the downloading rule request. The application current limiting module can perform current limiting control on the file in the uploading and downloading processes. The uploading and downloading service module can receive a file uploading request or a file downloading request, inquire metadata in the file uploading request, upload the newly added file and return uploading progress and address information of the newly added file; or inquiring the file address information in the file downloading request to execute the file downloading and return.
Fig. 10 is a schematic diagram of main blocks of a file data processing apparatus according to a fourth embodiment of the present invention, which may include an upload module 1001 and a download module 1002 as shown in fig. 10, applied to a client. The uploading module 1001 acquires attribute information of a file to be uploaded and client information, generates an uploading rule request and sends the uploading rule request to a server; receiving network point rule configuration information to perform fragmentation processing on a file to be uploaded, generating a file uploading request and sending the file uploading request to a server; and receiving and storing the uploading progress and the address information of the newly added file. The downloading module 1002 acquires attribute information of a file to be downloaded and client information, generates a downloading rule request and sends the downloading rule request to a server; and receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, sending the file downloading request to the server, and further receiving a file downloading result returned by the server. In some embodiments, the upload module 1001 is further configured to:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; and acquiring attribute information and client information of the file to be uploaded, and generating an uploading rule request based on the current limiting identifier and Token.
In some embodiments, after the uploading module 1001 receives and stores the uploading progress and the address information of the newly added file, the method includes:
and receiving a delayed retransmission instruction, positioning the fragmented file, and generating and sending a file uploading request according to the node rule configuration information.
In some embodiments, before the uploading module 1001 generates and sends the uploading rule request to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of an uploaded file;
and starting a file uploading plug-in according to the uploading file main switch parameter.
In some embodiments, the downloading module 1002 is further configured to:
sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; and acquiring attribute information and client information of the file to be downloaded, and generating a downloading rule request based on the current limiting identifier and Token.
In some embodiments, before the downloading module 1002 generates and sends the request of the downloading rule to the server, the method includes:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of a downloaded file;
and starting the file downloading plug-in according to the main switch parameter of the downloaded file.
As still some embodiments, as shown in fig. 11, the architecture of the client for the file uploading and downloading process includes a management module, an uploading module, a downloading module, and a UI interface module. The management module can be used for managing and storing the super-large file uploading/downloading control rules (namely website rule configuration information) of the client, and aiming at the service characteristics, file management and control can be realized according to lines, mechanisms, file menus, IP addresses, file types, file sizes, speed limit values and the like, so that the unified maintenance of relevant information of files is realized. In addition, the uploading module and the downloading module support file fragment management, namely, the uploaded or downloaded files are subjected to fragment processing; and supporting breakpoint continuous transmission in abnormal time, for example, positioning to a retransmitted initial fragment file for continuous transmission based on a delayed retransmission instruction; and file uploading or downloading is supported according to a speed limit value (namely, the transmission speed is determined) and the number of tasks (for example, at least one file fragment in the file uploading or file downloading process can be used as one task to carry out uploading or downloading control based on the number of tasks), so that normal business transaction of a website can not be influenced while uploading or downloading. And the UI interface module provides a visual file uploading and downloading interface, and comprises operable starting, pausing, deleting and the like functions.
As a preferred embodiment, as shown in fig. 12, the specific file uploading and downloading execution process based on the client and server architecture may include: the security module receives a Token request sent by a client, verifies an authorization secret key, after the verification is passed, the website current-limiting module calls the number of client website requests recorded in the cache, judges whether the number of the requests is larger than a preset concurrency threshold value, and if yes, returns a current-limiting identifier and generates a Token to the client through the security module. And if not, returning the generated Token through the security module.
The configuration module receives an uploading rule request or a downloading rule request sent by a client, verifies the Token in the uploading rule request or verifies the Token in the downloading rule request, and after the verification is passed, corresponding current limiting measures are adopted according to the current limiting identification so as to call the rule engine to generate website rule configuration information and return the website rule configuration information to the client based on the client information in the uploading rule request and the attribute information of the file to be uploaded, or the rule engine is called to generate the website rule configuration information based on the client information in the uploading rule request and the attribute information of the file to be downloaded.
And the uploading and downloading service module receives a file uploading request sent by the client according to the website rule configuration information, inquires metadata in the file uploading request so as to upload the newly added file, and returns uploading progress and newly added file address information. Wherein, the historical files can be positioned through the metadata, so as to upload the newly added files.
Or the uploading and downloading service module receives a file downloading request sent by the client according to the website rule configuration information, inquires file address information in the file downloading request, and executes file downloading and returns.
During the process of executing the uploading of the newly added file or executing the downloading of the file, the application current limiting module can call a token bucket algorithm to trigger an application current limiting program for uploading the newly added file or trigger an application current limiting program for downloading the file according to the rule configuration information of the network points. And if the transmission failure is monitored, triggering a delayed retransmission program to add the task into a timed retransmission queue.
Fig. 13 shows an exemplary system architecture 1300 to which a file data processing method or a file data processing apparatus of an embodiment of the present invention can be applied.
As shown in fig. 13, the system architecture 1300 may include terminal devices 1301, 1302, 1303, a network 1304, and a server 1305. The network 1304 is used to provide a medium for communication links between the terminal devices 1301, 1302, 1303 and the server 1305. The network 1304 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
Guests may interact with a server 1305 through a network 1304 using terminal devices 1301, 1302, 1303 to receive or send messages or the like. Various messaging client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (examples only) may be installed on the terminal devices 1301, 1302, 1303.
The terminal devices 1301, 1302, 1303 may be various electronic devices having a file data processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 1305 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by guests using the terminal devices 1301, 1302, 1 303. The backend management server may analyze and process the received data such as the product information query request, and feed back a processing result (for example, target push information and product information — just an example) to the terminal device.
It should be noted that the file data processing method provided by the embodiment of the present invention is generally executed by the server 1305, and accordingly, the computing apparatus is generally disposed in the server 1305.
It should be understood that the number of terminal devices, networks, and servers in fig. 13 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 14, shown is a block diagram of a computer system 1400 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 14 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present invention.
As shown in fig. 14, the computer system 1400 includes a Central Processing Unit (CPU) 1401 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1402 or a program loaded from a storage portion 1408 into a Random Access Memory (RAM) 1403. In the RAM1403, various programs and data necessary for the operation of the computer system 1400 are also stored. The CPU1401, ROM1402, and RAM1403 are connected to each other via a bus 1404. An input/output (I/O) interface 1405 is also connected to bus 1404.
The following components are connected to the I/O interface 1405: an input portion 1406 including a keyboard, a mouse, and the like; an output portion 1407 including a display such as a Cathode Ray Tube (CRT), a liquid crystal document data processor (LCD), and the like, and a speaker and the like; a storage portion 1408 including a hard disk and the like; and a communication portion 1409 including a network interface card such as a LAN card, a modem, or the like. The communication section 1409 performs communication processing via a network such as the internet. A drive 1410 is also connected to the I/O interface 1405 as needed. A removable medium 1411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1410 as necessary, so that a computer program read out therefrom is installed into the storage section 1408 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1409 and/or installed from the removable medium 1411. The computer program performs the above-described functions defined in the system of the present invention when executed by a Central Processing Unit (CPU) 1401.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a configuration module and an upload and download service module. Wherein the names of the modules do not in some way constitute a limitation on the modules themselves.
As another aspect, the present invention also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium bears one or more programs, and when the one or more programs are executed by the equipment, the equipment comprises a client-side receiving rule uploading request, a rule engine is called to generate website rule configuration information based on client-side information in the rule uploading request and attribute information of a file to be uploaded, and the website rule configuration information is returned; and receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request to upload the newly added file, and returning uploading progress and address information of the newly added file.
According to the technical scheme of the embodiment of the invention, the problems of low file management efficiency and poor security in the prior art can be solved.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (20)

1. A file data processing method is characterized in that the method is applied to a server and comprises the following steps:
receiving an uploading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded, and returning the website rule configuration information;
receiving a file uploading request sent by a client according to the website rule configuration information, inquiring metadata in the file uploading request to upload a newly added file, and returning uploading progress and address information of the newly added file;
receiving a Token request sent by a client, verifying an authorization secret key, and calling the number of client website requests recorded in a cache after the authorization secret key passes verification;
judging whether the request quantity is larger than a preset concurrency threshold value or not, if so, returning a current-limiting identifier and a generated Token, receiving an uploading rule request sent by a client, verifying the Token in the uploading rule request, calling a rule engine according to the current-limiting identifier after passing verification, and generating website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded; and if not, returning the generated Token, receiving an uploading rule request sent by the client, verifying the Token in the uploading rule request, and calling a rule engine to generate website rule configuration information based on the client information in the uploading rule request and the attribute information of the file to be uploaded after the Token passes verification.
2. The method of claim 1, wherein performing the uploading of the added file comprises:
and calling a preset current limiting algorithm, and triggering an application current limiting program uploaded to the newly added file according to the network point rule configuration information.
3. The method of claim 2, wherein triggering the application throttling program for uploading the newly added file according to the network node rule configuration information comprises:
judging whether the file uploading request meets an access rule in the node rule configuration information;
if so, carrying out multi-thread file transmission according to bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and mesh point rule configuration information;
and if not, rejecting the file uploading request.
4. The method of claim 3, after the multi-threaded file transfer based on the bandwidth limitation, comprising:
and monitoring file transmission failure and triggering a delayed retransmission program.
5. The method of claim 1, wherein prior to receiving the upload rule request sent by the client, the method comprises:
and acquiring the target attribute of the client, determining that the file uploading function is allowed to be used, and then returning the uploading file master switch parameter to the client so as to start the file uploading plugin of the client.
6. A file data processing method is applied to a client and comprises the following steps:
acquiring attribute information and client information of a file to be uploaded, generating an uploading rule request and sending the uploading rule request to a server; the method for acquiring attribute information and client information of a file to be uploaded and generating an uploading rule request comprises the following steps: sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; acquiring attribute information and client information of a file to be uploaded, and generating an uploading rule request based on a current limiting identifier and Token;
receiving website rule configuration information to perform fragment processing on a file to be uploaded, generating a file uploading request and sending the file uploading request to a server;
and receiving and storing the uploading progress and the address information of the newly added file.
7. The method of claim 6, wherein after receiving and storing the uploading progress and the new file address information, the method comprises:
and receiving a delayed retransmission instruction, positioning the fragmented file, and further generating a file uploading request according to the node rule configuration information and sending the file uploading request.
8. The method of claim 6, wherein before generating and sending the upload rule request to the server, the method comprises:
acquiring a client target attribute and sending the client target attribute to a server so as to receive a total switch parameter of an uploaded file;
and starting a file uploading plug-in according to the uploading file main switch parameter.
9. A file data processing method is applied to a server and comprises the following steps:
receiving a downloading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information and attribute information of a file to be downloaded in the downloading rule request, and returning the website rule configuration information;
receiving a file downloading request sent by a client according to the website rule configuration information, and inquiring file address information in the file downloading request to execute file downloading and return;
receiving a Token request sent by a client, verifying an authorization secret key, and calling the number of client website requests recorded in a cache after the authorization secret key passes verification;
judging whether the number of the requests is greater than a preset concurrency threshold value, if so, returning a current-limiting identifier and a generated Token, receiving a downloading rule request sent by a client, verifying the Token in the downloading rule request, calling a rule engine according to the current-limiting identifier after the verification is passed, and generating website rule configuration information based on client information in the downloading rule request and attribute information of a file to be downloaded; and if not, returning the generated Token, receiving a download rule request sent by the client, verifying the Token in the download rule request, and calling a rule engine to generate website rule configuration information based on the client information in the download rule request and the attribute information of the file to be downloaded after the Token passes verification.
10. The method of claim 9, wherein performing a file download comprises:
and calling a preset current limiting algorithm, and triggering an application current limiting program for downloading the file according to the network point rule configuration information.
11. The method of claim 10, wherein triggering an application throttling program for file downloading according to the website rule configuration information comprises:
judging whether the file downloading request meets an access rule in the website rule configuration information;
if so, carrying out multi-thread file transmission according to bandwidth limitation, wherein the transmission speed is determined based on a current limiting algorithm and mesh point rule configuration information;
if not, rejecting the file downloading request.
12. The method of claim 9, wherein prior to receiving the request for the download rule sent by the client, the method comprises:
and acquiring the target attribute of the client, determining that the file downloading function is allowed to be used, and then returning the main switch parameter of the downloaded file to the client so as to start the file downloading plug-in of the client.
13. A file data processing method is applied to a client and comprises the following steps:
acquiring attribute information and client information of a file to be downloaded, generating a downloading rule request and sending the downloading rule request to a server; the method for acquiring attribute information and client information of a file to be downloaded and generating a downloading rule request comprises the following steps: sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; acquiring attribute information and client information of a file to be downloaded, and generating a downloading rule request based on a current limiting identifier and Token;
and receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, sending the file downloading request to the server, and further receiving a file downloading result returned by the server.
14. The method of claim 13, wherein before generating and sending the request for the download rule to the server, the method comprises:
acquiring a target attribute of a client and sending the target attribute to a server so as to receive a total switch parameter of a downloaded file;
and starting the file downloading plug-in according to the download file master switch parameter.
15. A file data processing device is applied to a server and comprises:
the configuration module is used for receiving an uploading rule request sent by a client, calling a rule engine to generate website rule configuration information based on client information in the uploading rule request and attribute information of a file to be uploaded, and returning the website rule configuration information; or the rule engine is called to generate website rule configuration information and return the website rule configuration information based on the client information in the downloading rule request and the attribute information of the file to be downloaded;
the uploading and downloading service module is used for receiving a file uploading request sent by the client according to the website rule configuration information, inquiring metadata in the file uploading request so as to upload a newly added file, and returning an uploading progress and address information of the newly added file; or the server is used for receiving a file downloading request sent by the client according to the website rule configuration information, inquiring file address information in the file downloading request, executing file downloading and returning;
the security module is used for receiving a Token request sent by the client and verifying the authorization key;
the network node current limiting module is used for calling the client network node request quantity recorded in the cache after the verification is passed; judging whether the number of the requests is larger than a preset concurrency threshold value, if so, returning a current-limiting identifier and the generated Token; and if not, returning the generated Token.
16. The apparatus of claim 15, comprising:
the application current limiting module is used for calling a preset current limiting algorithm and triggering an application current limiting program uploaded to the newly added file according to the network point rule configuration information; or calling a preset current limiting algorithm, and triggering an application current limiting program for downloading the file according to the network point rule configuration information.
17. A file data processing device applied to a client comprises:
the uploading module is used for acquiring attribute information and client information of a file to be uploaded, generating an uploading rule request and sending the uploading rule request to the server; receiving network point rule configuration information to perform fragmentation processing on a file to be uploaded, generating a file uploading request and sending the file uploading request to a server; receiving and storing the uploading progress and the address information of the newly added file; the method for acquiring attribute information and client information of a file to be uploaded and generating an uploading rule request comprises the following steps: sending a Token request to a server according to a preset authorization key so as to receive a current limiting identifier and a generated Token; acquiring attribute information and client information of a file to be uploaded, and generating an uploading rule request based on a current limiting identifier and Token;
the downloading module is used for acquiring attribute information and client information of a file to be downloaded, generating a downloading rule request and sending the downloading rule request to the server; and receiving the rule configuration information of the website, generating a file downloading request according to the file address information of the file to be downloaded, sending the file downloading request to the server, and further receiving a file downloading result returned by the server.
18. The apparatus of claim 17, further comprising:
the management module is used for storing the rule configuration information of the network nodes;
and the UI interface module is used for providing an interface for uploading or downloading files.
19. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-5 or 6-8 or 9-12 or 13, 14.
20. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1-5 or 6-8 or 9-12 or 13, 14.
CN202110777032.2A 2021-07-09 2021-07-09 File data processing method and device, electronic equipment and computer readable medium Active CN113517985B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110777032.2A CN113517985B (en) 2021-07-09 2021-07-09 File data processing method and device, electronic equipment and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110777032.2A CN113517985B (en) 2021-07-09 2021-07-09 File data processing method and device, electronic equipment and computer readable medium

Publications (2)

Publication Number Publication Date
CN113517985A CN113517985A (en) 2021-10-19
CN113517985B true CN113517985B (en) 2022-11-01

Family

ID=78066558

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110777032.2A Active CN113517985B (en) 2021-07-09 2021-07-09 File data processing method and device, electronic equipment and computer readable medium

Country Status (1)

Country Link
CN (1) CN113517985B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114650277B (en) * 2022-03-11 2023-10-31 中国工商银行股份有限公司 File transmission control method and device
CN114826733B (en) * 2022-04-19 2024-01-05 中国工商银行股份有限公司 File transmission method, device, system, equipment, medium and program product
CN116015955B (en) * 2023-01-04 2023-12-01 三峡高科信息技术有限责任公司 Configurable method for verifying validity security of uploading file in application system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102821119A (en) * 2011-06-10 2012-12-12 北大方正集团有限公司 Client end, method, system and equipment for uploading files
CN110417888A (en) * 2019-07-30 2019-11-05 中国工商银行股份有限公司 Flow control methods, volume control device and electronic equipment
WO2019222934A1 (en) * 2018-05-23 2019-11-28 优视科技新加坡有限公司 File processing method, apparatus and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014113430A1 (en) * 2014-09-17 2016-03-17 Bundesdruckerei Gmbh Distributed data storage using authorization tokens

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102821119A (en) * 2011-06-10 2012-12-12 北大方正集团有限公司 Client end, method, system and equipment for uploading files
WO2019222934A1 (en) * 2018-05-23 2019-11-28 优视科技新加坡有限公司 File processing method, apparatus and system
CN110417888A (en) * 2019-07-30 2019-11-05 中国工商银行股份有限公司 Flow control methods, volume control device and electronic equipment

Also Published As

Publication number Publication date
CN113517985A (en) 2021-10-19

Similar Documents

Publication Publication Date Title
CN113517985B (en) File data processing method and device, electronic equipment and computer readable medium
CN104219316A (en) Method and device for processing call request in distributed system
CN109391673B (en) Method, system and terminal equipment for managing update file
CN109918191B (en) Method and device for preventing frequency of service request
CN111427701A (en) Workflow engine system and business processing method
CN111478781B (en) Message broadcasting method and device
CN113381944A (en) System current limiting method, apparatus, electronic device, medium, and program product
CN111680328A (en) Data processing method and device, server and computer readable storage medium
CN113595927A (en) Method and device for processing mirror flow in bypass mode
US11463549B2 (en) Facilitating inter-proxy communication via an existing protocol
CN110324384B (en) Data pushing method and device
CN111767157A (en) Request processing method and device based on service grid
CN110071952B (en) Service call quantity control method and device
CN113364887A (en) File downloading method based on FTP, proxy server and system
CN113010238A (en) Permission determination method, device and system for micro application call interface
CN113742389A (en) Service processing method and device
CN112948138A (en) Method and device for processing message
CN112149392A (en) Rich text editing method and device
CN113765871B (en) Method and device for managing fort machine
CN115412326A (en) Abnormal flow detection method and device, electronic equipment and storage medium
CN112688982B (en) User request processing method and device
CN114374657A (en) Data processing method and device
CN110247847B (en) Method and device for back source routing between nodes
CN110019671B (en) Method and system for processing real-time message
CN113542324A (en) Message pushing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant