CN111596933A

CN111596933A - File processing method and device, electronic equipment and computer readable storage medium

Info

Publication number: CN111596933A
Application number: CN202010659124.6A
Authority: CN
Inventors: 杨小璐; 刘焕新
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-07-09
Filing date: 2020-07-09
Publication date: 2020-08-28

Abstract

The application provides a file processing method and device, electronic equipment and a computer readable storage medium, and relates to the field of data processing. The method comprises the following steps: when the event task aiming at a preset file library is monitored to exist in a preset assembly, the preset assembly is called to obtain the event task, then the event task aiming at the file library is executed, an execution result is generated when the event task is executed, and the execution result is stored based on a preset rule. Therefore, all event tasks executed aiming at the document library can be recorded through the execution result, and a user can determine the historical event tasks executed aiming at the document library and the related information of the user initiating the event tasks by inquiring the execution result, so that the source of the event tasks is tracked, and particularly when the document library has problems, the safety of the document library is ensured.

Description

File processing method and device, electronic equipment and computer readable storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a file processing method and apparatus, an electronic device, and a computer-readable storage medium.

Background

yum (Yellow dog update, Modified): the Fedora and RedHat and Shell front-end software Package Manager in CentOS, based on RPM (Red-Hat Package Manager) Package management, can automatically download RPM packages from a specified server and install, can automatically handle dependency relationships, and install all dependent software packages at once. yum Source, refers to the location where yum downloads the RPM pack is specified before using yum to run the installation files, this location is referred to as yum Source.

yum common construction modes of the source: directly mounting the software source mirror image to a specified directory, and then building http or ftp service to enable other machines to obtain an installation package; synchronizing mirror image resources from a software source of the open rsync service to the local by using a primary rsync (remote synchronization), and then building http (Hypertext Transfer Protocol) or ftp (File Transfer Protocol) service to enable other machines to obtain an installation package; after the software installation package is downloaded, the software installation package is placed under a designated directory, the directory is created into a yum indexable directory by means of createperepo (used for creating yum sources), and services such as http are provided for other servers.

However, this construction method has the following disadvantages:

downloading the installation package, and using createrepo to create yum index directory to realize the service mode of yum source, flexible but poor security, especially inside the cluster, when the installation package with problems is imported, the whole yum source is easily affected, and the source of the problems cannot be traced because the operation cannot be recorded.

Disclosure of Invention

The application provides a file processing method, a file processing device, an electronic device and a computer readable storage medium, which can solve the problems. The technical scheme is as follows:

in a first aspect, a file processing method is provided, and the method includes:

when the fact that an event task aiming at a preset file library exists in a preset component is monitored, the preset component is called to obtain the event task; the event task is generated by a preset file management system in response to a processing request aiming at any file in the file library, and is issued to the preset component;

executing the event task aiming at the file library, and generating an execution result when the event task is completed;

and storing the execution result based on a preset rule.

Preferably, the method further comprises:

acquiring preset information of the file library; the preset information comprises the size, the port state and the process state of the file library;

and calling a preset warning component to detect the preset information, so that when the warning component detects that the preset information meets a preset warning condition, warning information is generated.

Preferably, the event task includes a first update task initiated by a user for a preset synchronization configuration file in the file library;

the executing the event task aiming at the file library and generating an execution result when the event task is completed comprises the following steps:

acquiring the synchronous configuration file from the file library, and updating the synchronous configuration file based on the first updating task;

if the updating is successful, generating an execution result of the successful updating based on the relevant information of the user; if the updating fails, generating an execution result of the updating failure based on the relevant information of the user; the related information of the user comprises at least one of user identity identification, IP address and position information.

Preferably, the event task comprises a second update task initiated by the user and aiming at any target file in the file library;

acquiring the target file from the file library, and updating the target file based on the second updating task;

if the updating is successful, generating an execution result of the successful updating based on the relevant information of the user; and if the updating fails, generating an execution result of the updating failure based on the relevant information of the user.

Preferably, the event task comprises a synchronization task of synchronizing a new file to the file repository;

based on the synchronization task, calling a preset synchronization component to acquire the new file from a preset remote file library, and storing the new file to the file library;

if the storage is successful, generating an execution result of successful synchronization; and if the storage fails, generating an execution result of synchronization failure.

Preferably, the synchronization task is generated by the file management system based on a synchronization request of a user, or generated based on a preset synchronization configuration file.

Preferably, the method further comprises the following steps:

when a preset first file component for downloading files is called to receive a downloading request aiming at any file in the file library, which is initiated by a user through a client, the first file component is called to send the any file to the client.

Preferably, the method further comprises the following steps:

when a preset second file component for uploading files is called to receive an uploading request for the files to be uploaded, which is initiated by a user, the second file component is called to receive the files to be uploaded, and the files to be uploaded are stored in the file library.

Preferably, the storing the execution result by using a preset rule includes:

storing the execution result into a preset information database through a preset file management system; the information database includes at least one execution result.

In a second aspect, there is provided a document processing apparatus comprising:

the first processing module is used for monitoring the existence of event tasks aiming at a preset file library in the preset assembly; the event task is generated by a preset file management system in response to a processing request aiming at any file in the file library, and is issued to the preset component;

the second processing module is used for calling the preset component to acquire the event task;

the third processing module is used for executing the event task aiming at the file library and generating an execution result when the event task is completed;

and the fourth processing module is used for storing the execution result based on a preset rule.

Preferably, the method further comprises:

the fifth processing module is used for acquiring preset information of the file library; the preset information comprises the size, the port state and the process state of the file library;

the sixth processing module is configured to invoke a preset warning component to detect the preset information, so that when the warning component detects that the preset information meets a preset warning condition, warning information is generated.

the third processing module comprises:

the first obtaining sub-module is used for obtaining the synchronous configuration file from the file library;

a first update sub-module, configured to update the synchronization configuration file based on the first update task;

and the first generation submodule is used for generating an execution result of successful update based on the relevant information of the user, or generating an execution result of failed update based on the relevant information of the user.

the third processing module comprises:

the second obtaining sub-module is used for obtaining the target file from the file library;

the second updating submodule is used for updating the target file based on the second updating task;

the second generation submodule is used for generating an execution result of successful update based on the relevant information of the user, or generating an execution result of failed update based on the relevant information of the user; the related information of the user comprises at least one of user identity identification, IP address and position information.

the third processing module comprises:

a third obtaining sub-module, configured to call a preset synchronization component to obtain the new file from a preset remote file library based on the synchronization task;

the storage submodule is used for storing the new file to the file library;

and the third generation submodule is used for generating an execution result of successful synchronization or an execution result of failed synchronization.

Preferably, the method further comprises the following steps:

the seventh processing module is used for calling a preset first file component for downloading a file to send any file to the client when a downloading request which is initiated by a user through the client and aims at any file in the file library is received.

Preferably, the method further comprises the following steps:

and the eighth processing module is used for calling the second file assembly to receive the file to be uploaded and storing the file to be uploaded to the file library when calling the preset second file assembly for uploading the file to receive an uploading request for the file to be uploaded, which is initiated by a user.

Preferably, the fourth processing module is specifically configured to:

In a third aspect, a computing device is provided, comprising: the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the file processing method as shown in the first aspect of the application.

In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the file processing method shown in the first aspect of the present application.

The beneficial effect that technical scheme that this application provided brought is:

when the event task aiming at a preset file library exists in a preset component, calling the preset component to acquire the event task, wherein the event task is generated by a preset file management system in response to a processing request aiming at any file in the file library, and issuing the event task to the preset component; and then executing the event task aiming at the file library, generating an execution result when the event task is completed, and storing the execution result based on a preset rule. Therefore, all event tasks executed aiming at the document library can be recorded through the execution result, and a user can determine the historical event tasks executed aiming at the document library and the related information of the user initiating the event tasks by inquiring the execution result, so that the source of the event tasks is tracked, and particularly when the document library has problems, the safety of the document library is ensured.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a general framework diagram of the present application;

FIG. 2 is a flowchart illustrating a document processing method according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a document processing method according to another embodiment of the present application;

FIG. 4 is a data structure diagram of a zookeeper of the present application;

FIG. 5-1 is a schematic structural diagram of a document processing apparatus according to another embodiment of the present application;

FIG. 5-2 is a schematic structural diagram of a document processing apparatus according to another embodiment of the present application;

5-3 are schematic structural diagrams of a document processing apparatus according to another embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device for processing a file according to yet another embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The terms referred to in this application will first be introduced and explained:

zookeeper: zookeeper is a distributed, open-source distributed application coordination service, is an open-source implementation of Chubby of Google, and is an important component of Hadoop and Hbase. It is a software that provides a consistent service for distributed applications, and the functions provided include: configuration maintenance, domain name service, distributed synchronization, group service, etc.

fileserver: file server refers to a file storage device accessible by all users in a computer network environment. File servers allow users to share information over a network without physically moving files through a floppy disk or some other external storage device. Any computer can be hosted and operate as a file server.

Fig. 1 is a schematic diagram of the overall framework of the present application. The repository may be, among other things, an yum source for storing installation files and synchronization configuration files for applications. By yum source, it is meant that before running the installation file using yum, the location where yum downloads the RPM package is specified, referred to as yum source. In other words, the yum source refers to the source from which the file is installed. At least one yum source is needed to install an application using yum, and the yum source can be either a network yum source or a local yum source.

Further, the document library may be one document library or a set including a plurality of sub document libraries, and may be set according to actual requirements in actual applications, which is not limited in the embodiment of the present invention.

An agent component: monitoring a zookeeper; acquiring an event task from a zookeeper and executing the event task; reporting the execution result to a file management system; the repository is monitored and the monitoring data is sent to the alert component. Wherein, the agent component can be agent and the warning component can be argus.

A synchronization component: and based on the synchronization task, synchronizing the new file in the remote file library into the file library, and reporting the execution result of the synchronization task to the file management system. Wherein the remote repository may be any repository other than a repository.

A first file component: and is responsible for providing file downloading services for the front end, such as providing file downloading services for users. Wherein, the first file component may be nginx.

A second file component: and the system is responsible for providing file uploading service for the front end, such as file uploading initiated by a user through a client. Wherein the second file component may be a fileserver agent.

The file management system may be yum-oss (Operation Support Systems), yum-oss responsible for:

1) receiving a request of a front end, such as a request initiated by a user, or a request initiated by an information database, and the like; generating an event task based on the request; sending the event task to a zookeeper; recording the information data to an information database;

2) receiving the execution result returned by the agent component; the information database is updated based on the execution result.

An information database: may be used to store synchronization configuration information for synchronizing files including, but not limited to, synchronization mode, higher level source (the destination remote repository for obtaining new files), synchronization period, etc. And the method can also be used for storing the execution result after the execution event task is completed.

It should be noted that the file library, the information database, the file management system, and each component may be deployed in one device, or may be deployed in different devices, for example, the file library, the proxy component, the synchronization component, the first file component, and the second file component are deployed in a current device, and the other parts are deployed in another device. In practical application, the setting can be performed according to practical requirements, and the application is not limited to this.

For convenience of description, the present application will be described in detail by taking an example in which the file library, the agent component, the synchronization component, the first file component, and the second file component are deployed in one device, and the other parts are deployed in another device.

The application provides a file processing method, a file processing device, an electronic device and a computer-readable storage medium, which aim to solve the above technical problems in the prior art.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

In one embodiment, a file processing method is provided, as shown in fig. 2, the method including:

step S201, when an event task aiming at a preset file library exists in a preset component, calling the preset component to obtain the event task; the event task is generated by a preset file management system in response to a processing request aiming at any file in a file library, and is issued to a preset component;

specifically, the preset component may be a zookeeper, which is used to provide a consistency service for the distributed application; listening to the preset component may be an agent component. When the calling broker component listens that the zookeeper stores an event task for the library of files, the broker component may call zookeeper to retrieve the event task. Wherein, the event task may include but is not limited to: the method comprises a first updating task initiated by a user and aiming at a preset synchronization configuration file in a file library, a second updating task initiated by the user and aiming at any target file in the file library, and a synchronization task for synchronizing a new file to the file library.

In practical application, it may also happen that event tasks respectively for a plurality of different target files are monitored at the same time, and then the agent component may obtain a plurality of event tasks at the same time, which may be set according to practical requirements in practical application, which is not limited in the embodiment of the present invention.

Further, the event task is generated by the preset file management system yum-oss responding to a processing request for any file in the file library, the processing request may be initiated by a user or an information database, and after the file management system generates the event task based on the processing request, the event task may be issued to the preset component zookeeper.

Step S202, executing an event task aiming at the file library, and generating an execution result when the event task is executed and completed;

and after the event task is acquired by calling the proxy component, the event task aiming at the file library can be executed, and after the execution of the event task is finished, an execution result is generated.

Step S203, storing the execution result based on the preset rule.

After the execution results are generated, the execution results may be stored for subsequent queries.

In the embodiment of the invention, when an event task aiming at a preset file library exists in a preset component, the preset component is called to obtain the event task, wherein the event task is generated by a preset file management system in response to a processing request aiming at any file in the file library, and the event task is issued to the preset component; and then executing the event task aiming at the file library, generating an execution result when the event task is completed, and storing the execution result based on a preset rule. Therefore, all event tasks executed aiming at the document library can be recorded through the execution result, and a user can determine the historical event tasks executed aiming at the document library and the related information of the user initiating the event tasks by inquiring the execution result, so that the source of the event tasks is tracked, and particularly when the document library has problems, the safety of the document library is ensured.

In another embodiment, a file processing method is provided, as shown in fig. 3, the method including:

step S301, when an event task aiming at a preset file library exists in a preset component, calling the preset component to acquire the event task; the event task is generated by a preset file management system in response to a processing request aiming at any file in a file library, and is issued to a preset component;

specifically, the preset component may be a zookeeper and the listening preset component may be a proxy component. When the calling broker component listens that the zookeeper stores an event task for the library of files, the broker component may call zookeeper to retrieve the event task. Wherein, the event task may include but is not limited to: the method comprises a first updating task initiated by a user and aiming at a preset synchronization configuration file in a file library, a second updating task initiated by the user and aiming at any target file in the file library, and a synchronization task for synchronizing a new file to the file library.

Further, the event task is a preset file management system, that is, yum-oss responds to a processing request for any file in the file library to generate, where the processing request may be initiated by a user or an information database, and after the file management system generates the event task based on the processing request, the event task may be issued to a preset component zookeeper.

Step S302, executing an event task aiming at the file library, and generating an execution result when the event task is executed and completed;

In a preferred embodiment of the present invention, executing an event task for a document library and generating an execution result when executing a completion event task includes:

acquiring a synchronous configuration file from a file library, and updating the synchronous configuration file based on a first updating task;

Specifically, a user may initiate an update request for the synchronization configuration file through a client, the client sends the update request to the file management system, the file management system issues an update task to the zookeeper after receiving the update request, and the update task may include an update instruction for the synchronization configuration file.

After the agent component obtains the update task, the agent component may first obtain the synchronization configuration file from the file library, and then update the synchronization configuration file according to the update instruction in the update task, for example, update the synchronization mode, update the upper-level source of synchronization, update the synchronization period, and the like. If the updating is successful, generating an execution result of the successful updating based on the relevant information of the user; and if the updating fails, generating an execution result of the updating failure based on the relevant information of the user. The relevant information of the user includes but is not limited to: user identification, IP address, location information, etc.; the results of the execution include, but are not limited to: user's relevant information, update date, update time, update content, etc., results of the update, repository identification, target file identification, etc.

For example, the execution result of successful update may be { zhang; 6 months and 10 days in 2020; 18: 06; a document repository 002; the synchronization period is modified to 3 days; success, the execution result of update failure may be { zhang san; 6 months and 10 days in 2020; 18: 06; a document repository 002; the synchronization period is modified to 3 days; failure }.

acquiring a target file from the file library, and updating the target file based on a second updating task;

Specifically, a user can initiate an update request for any target file in the file library through a client, the client sends the update request to the file management system, the file management system sends an update task to the zookeeper after receiving the update request, and the update task may include an update instruction for the target file.

After the agent component obtains the update task, the agent component may first obtain the target file from the file library, and then update the target file according to the update instruction in the update task, for example, modify the target file, move the target file, delete the target file, and the like. If the updating is successful, generating an execution result of the successful updating based on the relevant information of the user; and if the updating fails, generating an execution result of the updating failure based on the relevant information of the user. The relevant information of the user includes but is not limited to: user identification, IP address, location information, etc.; the results of the execution include, but are not limited to: user's relevant information, update date, update time, update content, etc., results of the update, repository identification, target file identification, etc.

For example, the execution result of successful update may be { zhang; 6 months and 10 days in 2020; 18: 06; a document repository 002; delete file 0035; success, the execution result of update failure may be { zhang san; 6 months and 10 days in 2020; 18: 06; a document repository 002; delete file 0035; failure }.

Further, in practical applications, the file library may include a folder in addition to the file, and the update task for the file is also applicable to the folder, which is not described herein again.

based on the synchronization task, calling a preset synchronization component to acquire a new file from a preset remote file library, and storing the new file to the file library;

Specifically, since the synchronization period is recorded in the synchronization configuration file, when the synchronization period is reached, the proxy component may start a synchronization task, and then call the synchronization component to acquire a new file from the remote file repository according to the synchronization configuration file, and store the acquired new file in the file repository.

If the storage is successful, generating an execution result of successful synchronization; and if the storage fails, generating an execution result of synchronization failure. Wherein, the execution result includes but is not limited to: synchronization date, synchronization time, synchronization type, synchronization mode, synchronized repository identification, synchronized file identification, result of synchronization, and the like.

For example, the execution result of successful synchronization may be { 10/6/2020; 18: 06; rsync synchronization; rsync; a document repository 002; the synchronization file 1035; success }; the execution result of synchronization failure may be {2020, 6, 10; 18: 06; self-defining synchronization; self-defining; a document repository 002; the synchronization file 1035; failure }.

Further, the synchronization types in the embodiment of the present invention may include rsync synchronization, custom synchronization, and manual synchronization. The synchronization configuration file adopted by the rsync synchronization can be an instruction list comprising at least one rsync instruction, and each rsync instruction in the instruction list can be executed when the rsync is synchronized; the synchronous configuration file adopted by the user-defined synchronization can be a script file comprising at least one script instruction, and all scripts in the script file can be executed during the user-defined synchronization; the manual synchronization is a synchronization task initiated by a user, the synchronization task comprises specific synchronization, and the automatic synchronization has a synchronization period, so that the user can initiate the synchronization task by himself before the next synchronization period is reached.

Specifically, a user may initiate a synchronization request through a client, the client sends the synchronization request to a file management system, the file management system issues a synchronization task to the zookeeper after receiving the synchronization request, and the synchronization task may include a start instruction for starting synchronization.

After the agent component obtains the synchronization task, the agent component starts the synchronization task according to a starting instruction in the synchronization task, then calls the synchronization component to obtain a preset synchronization configuration file from the file library, obtains a new file from the remote file library according to the synchronization configuration file, and stores the new file into the file library.

Due to manual synchronization, after synchronization is completed, an execution result may be generated based on the relevant information of the user, for example, an execution result of successful synchronization may be { zhang; 6 months and 10 days in 2020; 18: 06; manual synchronization; rsync; a document repository 002; the synchronization file 1035; success }.

Step S303, storing an execution result based on a preset rule;

In a preferred embodiment of the present invention, storing the execution result by using a preset rule includes:

and storing the execution result into a preset information database through a preset file management system.

Specifically, the execution result generated when the agent component executes the completion event task and the execution result generated after the synchronization component executes the completion event task may be transmitted back to the file management system, and then the file management system stores the execution task in the information database. Therefore, after the event task is executed and completed each time, the execution result can be stored in the information database for recording, and the user or the manager can know the executed and completed historical event task and the execution result by inquiring the execution result in the information database.

Further, since the information database also stores the synchronization configuration file (which is the same as the synchronization configuration file in the file database), if the event task is to modify the synchronization configuration file, the file management system, after receiving the execution result, can modify the synchronization configuration file in the information database in addition to storing the execution result in the information database, so that the synchronization configuration files of the two remain synchronized.

Step S304, acquiring preset information of a file library; the preset information comprises the size, the port state and the process state of the file library;

specifically, the agent component may perform an event task, and may also monitor the file library, and obtain preset information of the file library, where the preset information includes, but is not limited to: size of the file library, port state, process state, etc.

Further, the preset information for calling the agent component to obtain the file library may be obtained at preset time intervals, may also be obtained in real time, or may also be obtained in other manners, and may be set according to actual requirements in actual applications, which is not limited in this embodiment of the present invention.

Step S305, calling a preset warning component to detect preset information, so that when the warning component detects that the preset information meets the preset warning condition, warning information is generated;

specifically, after the calling agent component acquires the preset information, the warning component can be called and reports the preset information to the warning component, the warning component detects the reported warning information, if the preset information meets the preset warning condition, the warning information is generated, and then the warning information is output through an output interface of the warning component; and if the preset information does not meet the preset warning condition, finishing the detection.

Further, the warning component may store at least one warning condition configured in advance, and after the warning component receives the preset information reported by the agent component, the preset information may be detected based on each warning condition.

Step S306, when a preset first file component for downloading the file is called to receive a downloading request for any file in the file library, which is initiated by a user through a client, the first file component is called to send any file to the client;

specifically, the first file component can provide a file downloading service for the front end, so that a user can initiate a downloading request for any file in the file library to the first file component through the client, and after the first file component is called in the current device and the downloading request is received, the first file component is called to send the any file to the client, so that a downloading function is realized.

It should be noted that, in practical application, a user may initiate a download request for multiple files in a file library through a client at the same time, at this time, the current device may invoke the first file component to sequentially send the multiple files to the client, respectively, or send the multiple files to the client at the same time, and in practical application, the setting may be performed according to actual requirements, which is not limited in the embodiment of the present invention.

Further, in practical applications, after the user finishes downloading, the first file component may also report information about the user, file identification of the file downloaded by the user, download date, download time, and the like to the file management system, and the file management system records the information in the information database, so as to facilitate subsequent query.

In addition, the first file component may also provide a browsing service for the file repository. For example, the user can browse the files in the file library, the related information of the files, and the like through the first file component.

Step S307, when the preset second file component for uploading the file is called to receive an upload request for the file to be uploaded initiated by the user, the second file component is called to receive the file to be uploaded, and the file to be uploaded is stored in the file library.

Specifically, since the second file component can provide a file uploading service for the front end, the user can upload the file to be uploaded to the fileserver through the client, and the fileserver sends an uploading request for the file to be uploaded to the second file component and sends an uploading instruction to the file management system. The current device calls a second file component to receive the uploading request, and the file to be uploaded can be received from the fileserver after the uploading request is responded. When the fileserver starts to upload files, the file management system can monitor whether the file management system finishes uploading or not, can monitor in real time or at preset time intervals, and can record information such as user related information, file identification of the files uploaded by the user, uploading date and uploading time into the information database after uploading is finished so as to facilitate follow-up query.

Further, the file management system and the agent component perform data interaction through zookeeper, and the data structure of zookeeper can be as shown in fig. 4. Wherein, the data is transmitted according to the following protocol:

1. first level node

The yum-oss node is the storage root node for the entire repository of files.

2. Second level node

Each field of the second level node corresponds to a third level child node, and indicates whether the third level child node has a new update.

When yum-oss updates data of a third-level child node of zookeeper, setting a flag field of a second-level node corresponding to the child node to true (for example, if a configuration node related to configuration of the third-level node is updated, setting a configuration node of the second-level node above the configuration node to true); when the agent component agent receives the updated data of the third-level child node and completes the task according to the data, the flag field of the second-level node corresponding to the child node is set to false.

yum-oss with the second level node (i.e., the root node for each repository) as the status bit:

configuring a node: indicating whether the configuration node of the third level has update;

the instruction node: indicating whether the instruction node of the third level is updated;

file uploading node: and indicating whether the uploaded file node of the third level has update.

3. Third level node

1) Configuring a node

(1) The remote sync rsync includes: yum source name, yum source synchronization period, synchronization type, yum source synchronized upper level source, script to perform synchronization;

(2) the custom script comprises: yum source name, yum source synchronization period, synchronization type, script to perform synchronization;

(3) the manual uploading comprises the following steps: the synchronized script is executed.

2) Instruction node

The format is as follows: [ instruction ] [ parameter ] [ record ID of current task in synchronization history table ]; such as:

(1) newly creating a folder: a new folder centros/7 folder 1;

(2) renaming the file: rename centros/7/file _ name to centros/7/new _ file _ name;

(3) and (4) deleting the file: delete centros/7/file;

(4) enabling synchronization: the sync task of centros is started.

3) File uploading node

The format is as follows: UploadYumFile [ filename ] [ target path ] [ file upload task ID ] [ record ID of current task in synchronization history table ]

Further, in the prior art, the method of directly mounting the mirror image cannot realize the dynamic update of the software source mirror image in the yum source, and although the native remote synchronization method can realize the timing update by writing the remote synchronization command into the timing task, the configuration change is troublesome and easy to make mistakes, and it is difficult to monitor the synchronization condition. The embodiment of the invention presets the synchronous configuration file in the document library, can realize the dynamic update of the automatic document library by reading the synchronous configuration file, and the user can also update the document library based on the synchronous configuration file by initiating a synchronous task at any time, thereby realizing the dynamic update of the document library and ensuring the individuation of the update.

Furthermore, the agent component agent, the first file component nginx and the second file component fileserver agent are deployed in the current device to execute event tasks, file uploading and file downloading, so that not only can all requests of a user be processed, but also the agent component agent can monitor a file library, and abnormal warning of the file library is realized by combining a warning component argus. Meanwhile, the data interaction between the high-availability zookeeper and the agent component agent is adopted, so that the reliability of the data interaction is ensured.

Fig. 5-1 is a schematic structural diagram of a document processing apparatus according to another embodiment of the present application, and as shown in fig. 5-1, the apparatus according to this embodiment may include:

a first processing module 501, configured to monitor that an event task for a preset file library exists in a preset component; the event task is generated by a preset file management system in response to a processing request aiming at any file in a file library, and is issued to a preset component;

a second processing module 502, configured to invoke a preset component to obtain an event task;

a third processing module 503, configured to execute an event task for the file library, and generate an execution result when the event task is completed;

and a fourth processing module 504, configured to store the execution result based on a preset rule.

In a preferred embodiment of the present invention, as shown in fig. 5-2, the apparatus further comprises:

a fifth processing module 505, configured to obtain preset information of a file library; the preset information comprises the size, the port state and the process state of the file library;

a sixth processing module 506, configured to invoke a preset warning component to detect the preset information, so that when the warning component detects that the preset information meets a preset warning condition, warning information is generated.

In a preferred embodiment of the present invention, the event task includes a first update task initiated by a user and directed to a synchronization configuration file preset in a file library;

the third processing module comprises:

the first acquisition submodule is used for acquiring a synchronous configuration file from a file library;

the first updating submodule is used for updating the synchronous configuration file based on the first updating task;

the first generation submodule is used for generating an execution result of successful update based on the relevant information of the user or generating an execution result of failed update based on the relevant information of the user; the related information of the user comprises at least one of user identity identification, IP address and position information.

In a preferred embodiment of the present invention, the event task includes a second update task initiated by the user and directed to any target file in the file library;

the third processing module comprises:

the second acquisition submodule is used for acquiring a target file from the file library;

the second updating submodule is used for updating the target file based on a second updating task;

and the second generation submodule is used for generating an execution result of successful update based on the relevant information of the user or generating an execution result of failed update based on the relevant information of the user.

In a preferred embodiment of the invention, the event tasks include a synchronization task to synchronize a new file to the repository of files;

the third processing module comprises:

the third acquisition submodule is used for calling a preset synchronization component to acquire a new file from a preset remote file library based on the synchronization task;

the storage submodule is used for storing the new file to a file library;

In a preferred embodiment of the present invention, the synchronization task is generated by the file management system based on a synchronization request of a user, or based on a preset synchronization profile.

In a preferred embodiment of the present invention, as shown in fig. 5-3, further comprising:

the seventh processing module 507 is configured to, when a preset first file component for downloading a file is called to receive a download request for any file in the file library, initiated by a user through a client, call the first file component to send any file to the client.

The eighth processing module 508 is configured to, when a preset second file component for uploading a file is called to receive an upload request for a file to be uploaded, which is initiated by a user, call the second file component to receive the file to be uploaded, and store the file to be uploaded in the file library.

In a preferred embodiment of the present invention, the fourth processing module is specifically configured to:

The file processing apparatus of this embodiment can execute the file processing methods shown in the first embodiment and the second embodiment of this application, and the implementation principles thereof are similar, and are not described herein again.

Further, in the prior art, the method of directly mounting the mirror image cannot realize dynamic update of the software source mirror image in the yum source, and although the native rsync method can realize timing update by writing the rsync command into the timing task, configuration change is troublesome and prone to errors, and it is difficult to monitor synchronization conditions. The embodiment of the invention presets the synchronous configuration file in the document library, can realize the dynamic update of the automatic document library by reading the synchronous configuration file, and the user can also update the document library based on the synchronous configuration file by initiating a synchronous task at any time, thereby realizing the dynamic update of the document library and ensuring the individuation of the update.

Furthermore, the agent, the fileserver agent and the nginx are deployed in the current equipment to execute event tasks, file uploading and file downloading, so that not only can all requests of a user be processed, but also the agent can monitor the file library, and the abnormal warning of the file library is realized by combining argus. Meanwhile, the data interaction between the zookeeper and the agent is highly available, so that the reliability of the data interaction is ensured.

In another embodiment of the present application, there is provided an electronic device including: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: when the event task aiming at a preset file library exists in the preset component, calling the preset component to acquire the event task, wherein the event task is generated by a preset file management system in response to a processing request aiming at any file in the file library, and issuing the event task to the preset component; and then executing the event task aiming at the file library, generating an execution result when the event task is completed, and storing the execution result based on a preset rule. Therefore, all event tasks executed aiming at the document library can be recorded through the execution result, and a user can determine the historical event tasks executed aiming at the document library and the related information of the user initiating the event tasks by inquiring the execution result, so that the source of the event tasks is tracked, and particularly when the document library has problems, the safety of the document library is ensured.

In an alternative embodiment, an electronic device is provided, as shown in fig. 6, an electronic device 6000 shown in fig. 6 comprising: a processor 6001 and a memory 6003. Processor 6001 and memory 6003 are coupled, such as via bus 6002. Optionally, the electronic device 6000 may also include a transceiver 6004. It should be noted that the transceiver 6004 is not limited to one in practical applications, and the structure of the electronic device 6000 is not limited to the embodiment of the present application.

The processor 6001 could be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 6001 might also be a combination that performs a computing function, such as a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

The bus 6002 may include a path that conveys information between the aforementioned components. The bus 6002 may be a PCI bus, an EISA bus, or the like. The bus 6002 can be divided into an address bus, a data bus, a control bus, and so forth. For ease of illustration, only one thick line is shown in FIG. 6, but this is not intended to represent only one bus or type of bus.

Memory 6003 can be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

The memory 6003 is used to store application code that implements aspects of the subject application, and execution is controlled by the processor 6001. Processor 6001 is configured to execute application program code stored in memory 6003 to implement the teachings of any of the foregoing method embodiments.

Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like.

Yet another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, which, when run on a computer, enables the computer to perform the corresponding content in the aforementioned method embodiments. Compared with the prior art, when the event task aiming at the preset file library exists in the preset component, the preset component is called to obtain the event task, wherein the event task is generated by the preset file management system in response to a processing request aiming at any file in the file library, and the event task is issued to the preset component; and then executing the event task aiming at the file library, generating an execution result when the event task is completed, and storing the execution result based on a preset rule. Therefore, all event tasks executed aiming at the document library can be recorded through the execution result, and a user can determine the historical event tasks executed aiming at the document library and the related information of the user initiating the event tasks by inquiring the execution result, so that the source of the event tasks is tracked, and particularly when the document library has problems, the safety of the document library is ensured.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A file processing method, comprising:

and storing the execution result based on a preset rule.

2. The document processing method according to claim 1, characterized in that the method further comprises:

3. The file processing method according to claim 1 or 2, wherein the event task comprises a first update task initiated by a user for a synchronization configuration file preset in the file repository;

4. The file processing method according to claim 1 or 2, wherein the event task comprises a second update task initiated by a user for any target file in the file library;

5. The file processing method according to claim 1 or 2, wherein the event task includes a synchronization task of synchronizing a new file to the file repository;

6. The file processing method according to claim 5, wherein the synchronization task is generated by the file management system based on a synchronization request of a user or based on a preset synchronization profile.

7. The file processing method according to claim 1 or 2, further comprising:

8. The file processing method according to claim 1 or 2, further comprising:

9. The file processing method according to claim 1, 5 or 6, wherein the storing the execution result by using the preset rule comprises:

10. A document processing apparatus, characterized by comprising:

11. An electronic device, comprising:

a processor, a memory, and a bus;

the bus is used for connecting the processor and the memory;

the memory is used for storing operation instructions;

the processor is used for executing the file processing method of any one of the claims 1 to 9 by calling the operation instruction.

12. A computer-readable storage medium for storing computer instructions which, when executed on a computer, cause the computer to perform the file processing method of any one of claims 1 to 9.