CN114338654A

CN114338654A - File transmission method, storage service distribution method, device, equipment and medium

Info

Publication number: CN114338654A
Application number: CN202111664298.2A
Authority: CN
Inventors: 吴姗; 陶永; 杨景; 宁运飞
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2022-04-12

Abstract

In the file transmission method, the storage service allocation method, the device, the equipment and the medium provided in the embodiment of the application, the service device acquires and calculates capacity performance parameters of the storage service units, extracts corresponding transmission performance parameters according to the notification completed by the storage service units, combines the acquired accurate storage performance parameters of the storage service units, determines each storage service unit through a related preset storage service allocation rule to provide for the file transmission service units, the file transmission service units inquire the storage service units matched with the transmitted files in a candidate storage service list so as to complete the execution of a file transmission task, and can form a notification to feed back to the service device to form a closed loop, so that the effect of balanced storage service allocation is continuously enhanced, the storage performance is optimized, and conditions such as file transmission failure and high availability guarantee can be used for reference so as to eliminate unreliable storage service units, and the reliability of the storage service is improved.

Description

File transmission method, storage service distribution method, device, equipment and medium

Technical Field

The present application relates to the field of storage management technologies, and in particular, to a file transfer method, a storage service allocation method, an apparatus, a device, and a medium.

Background

The video monitoring platform mostly uses a distributed storage system as a video storage service, but object storage (OSS) is increasingly favored by the video monitoring platform as a video service due to its characteristics of high reliability and cost performance. In addition, the external media recording server may load hundreds of devices at the same time, and if only one storage service is accessed, a large amount of video may be lost if a failure occurs, so the monitoring platform may consider that the same storage server distributes multiple storage services.

However, the video files are characterized in that the generated time distribution may not be uniform, and the files have different sizes, which may cause some storage services to be in a busy state and some storage services to be in an idle state, thereby causing the storage services to have different usage spaces. The storage bandwidth is reduced and data is lost due to the fact that the space-free storage service cannot be used, and storage performance and reliability are reduced.

Inventing messages

In view of the above-mentioned shortcomings of the prior art, the present application provides a file transfer method, a storage service allocation method, apparatus, device and medium, which optimize the performance and reliability of a storage service by balancing the storage service.

The first aspect of the present application provides a file transmission method, which is applied to a file transmission service unit transmitting a video file to a storage service unit allocated by a service management device; the method comprises the following steps: the file transmission service unit puts the transmitted file into a file transmission queue according to the file transmission task; the file transmission service unit determines a target storage service unit for storing the transmitted file according to the candidate storage service list; wherein the candidate storage service list comprises a plurality of storage service units; each storage service unit is determined from each available storage service unit according to the storage performance parameters of each storage service unit by the service management device by using a preset storage service distribution rule and is provided to the file transmission service unit; the file transmission service unit completes a file transmission task by using the target storage service unit; and the file transmission service unit responds to the completion of the file transmission task and sends a transmission task execution success notice to the service management device, wherein the transmission task execution success notice comprises the transmission performance parameters of the corresponding target storage service unit and is used as a reference basis for calculating the storage performance parameters when the storage service unit is determined again.

In some embodiments, the file transfer service unit determining a destination storage service unit for storing the transferred file according to the candidate storage service list, including: determining whether the candidate storage service list meets a preset condition; wherein the preset conditions include: the storage service units available in the candidate storage service list are sufficient; and/or the non-updated time of the candidate storage service list exceeds a preset time threshold; in response to the preset condition being met, the file transmission service unit determines a storage service unit matched with the transmitted file in the candidate storage service list as a target storage service unit; in response to the preset condition not being satisfied, the file transfer service unit sending a candidate storage service allocation request to the service management apparatus to request allocation of the storage service unit; and determining a destination storage service unit based on the candidate storage service list updated corresponding to the allocation.

In some embodiments, the file transfer service unit determining a destination storage service unit for storing the transferred file according to the candidate storage service list, including: and avoiding selecting the storage service unit which has failed in the previous n times in the candidate storage service list as a target storage service unit, wherein n is more than or equal to 1.

In some embodiments, the file transfer method further includes: an exception handling action is performed based on the statistics of the transmission failure event.

In some embodiments, said performing exception handling actions based on statistics of transmission failure events comprises at least one of:

1) in response to the fact that the transmission failure times of the files exceed a first preset time threshold value, removing the files from a file transmission queue, adding the files into the file transmission failure queue, and sending a file transmission failure notice to a service management device;

2) and in response to the number of times of failure of continuous file transmission which has occurred to the candidate storage service unit exceeding a second preset number threshold, moving the candidate storage service unit out of the candidate storage service list, and sending a storage service exception notification to a service management device.

In some embodiments, the file transfer method includes: excluding the transmitted files with the file size exceeding a preset capacity threshold value from the file transmission queue; wherein the preset capacity threshold is associated with a maximum capacity of a storage service unit.

A second aspect of the present application provides a storage service allocation method, which is applied to allocate a storage service unit for a file transmission service unit to transmit a file; the method comprises the following steps: the service management device collects the storage use information of each storage service unit and calculates capacity performance parameters; the service management device extracts the transmission performance parameters of the corresponding target storage service unit according to the transmission task execution success notice which is received from the file transmission service unit and is about the completion of the file transmission task; the target storage service unit is used for storing the video files after transmission is finished; the service management device calculates storage performance parameters based on the transmission performance parameters and the capacity performance parameters corresponding to each storage service unit; the service management device determines candidate storage service units from all available storage service units based on the storage performance parameters of all the storage service units by using a preset allocation rule; the service management apparatus transmits information of the determined storage service unit to the file transfer service unit in response to the candidate service allocation request.

In some embodiments, the service management device calculates the storage performance parameter based on the transmission performance parameter and the capacity performance parameter corresponding to each storage service unit, including: and calculating the weighting result of the transmission performance parameter and the capacity performance parameter.

In some embodiments, the determining, by the service management apparatus, the storage service unit based on the storage performance parameter of each storage service unit by using a preset allocation rule includes: the service management device selects a preset distribution number of storage service units with higher storage performance parameters to be used for determining the storage service units.

In some embodiments, the service management apparatus determines the storage service units based on the storage performance parameters of the storage service units by using a preset allocation rule, and further includes: storage service units whose capacity performance parameters exceed a preset utilization threshold are excluded from allocation.

In some embodiments, the storage service allocation method includes: in response to a candidate storage service allocation request for a file transfer service unit, the service management apparatus determines a storage service unit in response to the candidate storage service allocation request.

In some embodiments, the candidate storage service allocation request is triggered by a preset condition not being satisfied, the preset condition comprising: the candidate storage service in the candidate storage service list of the file transmission service unit is enough to complete the file transmission task; and/or the non-update time of the candidate storage service list exceeds a preset time threshold.

In some embodiments, the storage service allocation method includes: in response to the number of available storage service units being less than a preset allocation number, the service management device takes each of the available storage service units as the candidate storage service unit.

In some embodiments, the storage service allocation method includes: in response to a storage exception notification indicating that a storage service unit is abnormal, the service management apparatus marks the corresponding storage service unit as abnormal to be restricted from being allocated; alternatively, the flag of the exception is removed in response to the storage service unit indicating the exception returning to normal.

A third aspect of the present application provides a file transmission apparatus, which is applied to a file transmission service unit for transmitting a video file to a storage service unit allocated by a service management apparatus; the device comprises: the transmission task execution module is used for putting the transmitted file into a file transmission queue according to the file transmission task; the destination storage determining module is used for determining a destination storage service unit for storing the transmitted file according to the candidate storage service list; wherein the candidate storage service list comprises a plurality of storage service units; each storage service unit is determined from each available storage service unit according to the storage performance parameters of each storage service unit by the service management device by using a preset storage service distribution rule and is provided to the file transmission service unit; the transmission task execution module is used for completing a file transmission task by using the target storage service unit; and the execution result reporting module is used for responding to the completion of the file transmission task and sending a transmission task execution success notice to the service management device, wherein the transmission task execution success notice comprises the transmission performance parameters of the corresponding target storage service unit and is used as a reference basis for calculating the storage performance parameters when the storage service unit is determined again.

A fourth aspect of the present application provides a storage service allocation apparatus, which is applied to allocate a storage service unit for a file transmission service unit to transmit a file; the device comprises: the storage information acquisition module is used for acquiring the storage use information of each storage service unit and calculating capacity performance parameters; the information extraction module is used for extracting the transmission performance parameters of the corresponding target storage service unit according to the transmission task execution success notice which is received from the file transmission service unit and is about the completion of the file transmission task; the storage performance calculation module is used for calculating storage performance parameters based on the transmission performance parameters and the capacity performance parameters corresponding to each storage service unit; the information generation module is used for determining candidate storage service units from all available storage service units based on the storage performance parameters of all the storage service units by utilizing a preset distribution rule; and the sending module is used for responding to the candidate service allocation request, and sending the information of the determined storage service unit to the file transmission service unit so as to determine a target storage service unit for storing the transmitted file in the file transmission task.

A fifth aspect of the present application provides a computer device comprising: a communicator, a memory, and a processor; the communicator is used for communicating with the outside; the memory is to store program instructions; the processor is configured to execute the program instructions to perform the file transfer method according to any one of the first aspect; alternatively, the storage service allocation method of any of the second aspects is performed.

A sixth aspect of the present application is a computer-readable storage medium storing program instructions that, when executed, perform the file transfer method according to any one of the first aspects; alternatively, the storage service allocation method of any of the second aspects is performed.

As described above, in the file transmission method, the storage service allocation method, the device, the apparatus, and the medium provided in this embodiment of the present application, the service device acquires and calculates the capacity performance parameters of the storage service units, extracts the corresponding transmission performance parameters according to the notification of the storage service units, combines the acquired storage performance parameters with the accurate storage service unit performance, and determines each storage service unit according to the relevant preset storage service allocation rule to provide the storage service unit for the file transmission service unit, and the file transmission service unit queries the storage service unit matching the transmitted file from the candidate storage service list, thereby completing the execution of the file transmission task, and can form a notification to feed back to the service device to form a closed loop, so that the effect of the storage service allocation balance is continuously enhanced, the storage performance is optimized, and the file transmission failure, and the storage performance of the file are also referred to, High availability guarantee and other conditions are used for eliminating unreliable storage service units and improving the reliability of the storage service. Therefore, the problem of uneven storage service use caused by the characteristics of inconsistent time distribution and different sizes of video files in the related technology can be effectively solved.

Drawings

Fig. 1 shows a schematic diagram of an application scenario in an embodiment of the present application.

Fig. 2 shows a flow chart of a storage service allocation method according to an embodiment of the present application.

Fig. 3 shows a flowchart of a file transfer method according to an embodiment of the present application.

Fig. 4 shows an interaction diagram of a file transfer process in an application example of the present application.

Fig. 5 shows a schematic flow chart of storage service allocation performed by the service management device in an application example of the present application.

Fig. 6 shows a block diagram of a file transfer device according to an embodiment of the present application.

Fig. 7 shows a block diagram of a storage service device according to an embodiment of the present application.

Fig. 8 shows a schematic structural diagram of a computer device in an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. The present application is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present application. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

Embodiments of the present application will be described in detail below with reference to the accompanying drawings so that those skilled in the art to which the present application pertains can easily carry out the present application. The present application may be embodied in many different forms and is not limited to the embodiments described herein.

Reference throughout this specification to "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," or the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics shown may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of different embodiments or examples presented in this application can be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first", "second" are used merely to denote an object and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the expressions of the present application, "plurality" means two or more unless specifically defined otherwise.

In order to clearly explain the present application, components that are not related to the description are omitted, and the same reference numerals are given to the same or similar components throughout the specification.

Throughout the specification, when a device is referred to as being "connected" to another device, this includes not only the case of being "directly connected" but also the case of being "indirectly connected" with another element interposed therebetween. In addition, when a device "includes" a certain component, unless otherwise stated, the device does not exclude other components, but may include other components.

Although the terms first, second, etc. may be used herein to refer to various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first interface and the second interface are represented. Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, steps, operations, elements, modules, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, steps, operations, elements, modules, items, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some way.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" include plural forms as long as the words do not expressly indicate a contrary meaning. The term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, regions, integers, steps, operations, elements, and/or components, but does not exclude the presence or addition of other features, regions, integers, steps, operations, elements, and/or components.

Although not defined differently, including technical and scientific terms used herein, all terms have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. Terms defined in commonly used dictionaries are to be additionally interpreted as having meanings consistent with those of related art documents and currently prompted messages, and should not be excessively interpreted as having ideal or very formulaic meanings unless defined.

Currently, in the related video security technology, a video file is uploaded through a multi-storage service. However, due to the diversity of scenes corresponding to video security, the generation time distribution of the video files is not uniform, and the file sizes are not uniform, which causes the storage resource usage of the storage service to be extremely non-uniform, thereby causing the problem that the storage performance and reliability of the multi-storage service are greatly affected.

In view of this, the embodiments of the present application may provide a corresponding solution to balance the usage of storage services.

It should be noted that the "units" referred to below, such as "storage service unit", "file upload service unit", and the like, indicate functional units that may be formed by a part of software and/or hardware resources in a certain hardware device (such as a server), for example, the storage service unit includes a storage resource (such as a cache) for storing a file. It is possible that the "unit" may also have a task thread configured to implement the respective function, which is responsible for executing the corresponding task.

In addition, although "transmission" referred to herein, such as "transmitted file", "file transmission service unit", etc., is shown as an "uploading" example because there is a difference in the identities of the "terminal" and the "server", it is understood that in other embodiments, the "downloading" example may be applied, or only transmission between two ends with the same identity, etc., and is not limited by way of example.

Fig. 1 is a schematic diagram of an application scenario in an embodiment of the present application.

In fig. 1, n streaming media service units, streaming media service unit #1 to streaming media service unit # n, are exemplarily shown. In a possible embodiment, the n streaming media service units #1 may be distributed over a plurality of hardware devices. And, n storage service units, storage service unit #1 to storage service unit # n, are presented.

Each streaming media service unit may include a video save service unit, a file transfer queue, a file transfer service unit, and a video file cache. The figure representatively shows an exemplary structure of a streaming media service # 1. Illustratively, the video saving service unit is configured to receive a video stream sent by the monitoring device and save the generated video file. The method can store the video files in the video file cache, and can add the file information (such as identification information and characteristic information) representing the video files into a file transmission queue. Illustratively, the file transmission service unit is configured to correspondingly upload the video files in each video file cache to a remote end according to the sequence of the file information in the file transmission queue and store the video files in the storage service unit, so as to complete file transmission. Optionally, the file transmission service unit may further perform exception handling according to the transmission failure condition and notify the service management apparatus, for example, put the file that fails to be transmitted multiple times into the file transmission failure queue and send an exception notification of file transmission failure to the service management apparatus, or perform exception notification also to the storage service unit that fails to be transmitted multiple times continuously. The file transfer service unit may maintain a candidate storage service list in which information of each storage service unit is stored, so as to be selected as the storage service unit for executing the file transfer. The storage service unit is determined by the service management apparatus and provides information to the file transfer service unit.

In some embodiments, the amount of data transmitted for each file transmission task may be part of a video file, a complete video file, parts of multiple video files, multiple complete video files, or parts of one or more complete video files and one or more video files, and may be selected according to actual needs.

Also exemplarily shown in fig. 1 is a service management apparatus. In some embodiments, the service management apparatus may be implemented as a central service management server or server group.

In some embodiments, the service management apparatus may determine information of each candidate storage service unit of the next period from the storage service units #1 to # n in response to the candidate storage service allocation request of the file transfer service unit and provide the information to the file transfer service unit, so that the file transfer service unit updates the candidate storage service list, and selects a destination storage service unit matching the video file in the list as a "destination" for storing after file transfer when file transfer is performed, so as to complete a file transfer task.

The service management means may determine a plurality of storage service units from among the respective storage service units according to a preset storage service allocation rule. Illustratively, the basis of the predicted storage service allocation rule may refer to the performance of the storage service unit only, or may also consider the storage performance and reliability (i.e. high availability) of the storage service unit.

On one hand, the service management device can collect and evaluate the storage performance of each storage service unit, and preferentially selects the storage service unit with better storage performance as the storage service unit under the load balancing principle. For example, the storage performance may calculate a capacity performance parameter by periodically collecting capacity usage information of each storage service unit, and on the other hand, may calculate a storage performance parameter of each storage service unit by weighting the storage performance parameter according to the transmission performance of the video file that is transmitted by each storage service unit, for example, using the transmission performance parameter as an evaluation parameter, and according to the capacity performance parameter and the transmission performance parameter, select a preset allocation number of storage service units in which the storage performance parameter is relatively higher, so as to be used for selecting the storage service units.

On the other hand, the service management device receives the request or the notice sent by each file transmission service unit and carries out corresponding processing. In a possible example, the service management device may receive one or more of a notification that the transmission task reported by each file transmission service unit is successfully executed (which may indicate that all video file data corresponding to a certain task is successfully uploaded), an exception notification related to file exception (which may indicate that multiple file transmissions fail), a storage service unit request related to storage service exception (for example, an available storage service unit is not enough to complete file transmission, and information of the storage service unit is unreliable), and the like, and the service management device may refer to information in the "exception" notification to cooperate with a balancing principle to exclude an inappropriate storage service unit as a storage service unit, so as to meet reliability (i.e., high availability) requirements.

It should be noted that the architecture for transmitting and storing the video files shown in the application scenario of fig. 1 and the above list of some notifications are only examples, and all of them can be changed according to actual requirements, for example, in the case of taking storage performance or reliability as the first place, the selection criteria of the storage service unit (i.e. the content of the "preset storage service distribution rule") can be changed. The number of "units" and the corresponding number of hardware devices may also be selected according to actual needs in view of system operation performance.

Fig. 2 is a schematic flow chart showing a storage service allocation method according to an embodiment of the present application. The storage service allocation method can be applied to a service management apparatus.

The process comprises the following steps:

step S201: the service management device collects the storage use information of each storage service unit and calculates the capacity performance parameters.

In some embodiments, the service management apparatus may periodically (for example, T1 is a period duration) collect information such as total capacity and used capacity of each storage service unit, and then calculate storage utilization as a capacity performance parameter, for example, the used capacity is compared with the total capacity.

Step S202: the service management device extracts the transmission performance parameters of the corresponding target storage service unit according to the transmission task execution success notice which is received from the file transmission service unit and is about the completion of the file transmission task.

In order to more accurately evaluate the storage performance of the storage service unit during load balancing, the capacity performance parameter may serve as one of reference bases for evaluating the storage performance of the storage service unit, and the transmission performance of the storage service unit during the execution of the file transmission task may also serve as another reference base for evaluating the storage performance of the storage service unit, where the transmission performance is expressed by the transmission performance acceptance number. Illustratively, the transmission performance parameter may be a file transmission rate; alternatively, the file transfer rate may be an average transfer rate of each destination storage service unit used when the entire file transfer task is completed.

Step S203: the service management device calculates the storage performance parameters based on the transmission performance parameters and the capacity performance parameters corresponding to each storage service unit.

In some embodiments, the storage performance parameter may be obtained by calculating a weighted sum of the transmission performance parameter and the capacity performance parameter. The capacity performance parameter is 0-1, the transmission performance parameter is in the order of magnitude of MB/s, and the weighted results can accurately reflect the accurate contribution of the capacity performance parameter and the transmission performance parameter to the storage performance parameter by reasonably setting the weighting coefficients of the capacity performance parameter and the transmission performance parameter.

In some embodiments, storage service units that are not suitable for use may be excluded as storage service units by the capacity performance parameters before calculating the storage performance parameters, which is to meet the requirement of "high availability". For example, the capacity performance parameter of a certain storage service unit is greater than a preset utilization threshold (e.g., 70%, 80%, 90%, etc.), which indicates that the capacity performance parameter of the storage service unit is too high to be used for storing the video file again.

Step S204: the service management device determines candidate storage service units from the available storage service units based on the storage performance parameters of the storage service units by using a preset allocation rule.

In some embodiments, the preset allocation rules may include load balancing rules related to storage service unit performance. Accordingly, after calculating each storage service unit, the service management device may select a preset allocation number of storage service units with a higher storage performance parameter for determining the storage service units therefrom. In a possible example, the storage service units with the preset allocation number at the top can be selected according to the storage performance parameters sorted from high to low.

In some embodiments, the preset allocation rules may further include reliability rules relating to storage service unit reliability (i.e., high availability). The service management means may mark the corresponding previously determined storage service unit as abnormal to be continuously allocated in the next round according to a storage abnormality notification generated by a file transfer failure sent from the file transfer service unit. Alternatively, when the storage service unit receiving the exception returns to normal (e.g., triggered by a manual repair), the exception flag is removed and assignable is resumed.

Step S205: the service management apparatus transmits information of the determined storage service unit to the file transfer service unit in response to the candidate service allocation request.

The allocation of the storage service unit to the file transfer service unit by the service management apparatus may be passively performed triggered by a preset condition. When the storage service units in the candidate storage service list cannot meet the requirement of the file transfer service unit, for example, the storage resource capacity corresponding to the storage service unit is smaller than the data volume of the file to be transferred, or if the time of the candidate service list which is not updated continuously exceeds a preset time threshold, the reliability of the storage service unit therein cannot be guaranteed, and the file transfer service unit may provide a candidate storage service allocation request to the service management apparatus to request allocation of the storage service unit. The service management apparatus may perform the above-described storage service allocation method to determine a new storage service unit to allocate to the requesting file transfer service unit according to the candidate storage service allocation request. In still other embodiments, the service management device may also periodically and actively execute the storage service allocation method to update its candidate storage service list for the file transfer service unit.

In some embodiments, the storage service allocation method may further include: in response to a storage exception notification indicating that a storage service unit is abnormal, the service management apparatus marks the corresponding storage service unit as abnormal to be restricted from being allocated, and may be ignored at the time of selecting the storage service unit of the next round; alternatively, the flag of the exception is removed in response to the storage service unit indicating the exception returning to normal.

Wherein the "exception" may be generated based on a statistical result of a video file upload failure event. In some examples, if file upload failures occur continuously, it may be that the file itself has a problem, for example, that the same file fails to upload repeatedly more than a first preset number threshold; there may be a problem of the storage service unit, for example, a certain storage service unit fails in a plurality of consecutive file transfers and exceeds a preset time such as a second preset number threshold.

In light of the foregoing, in some specific embodiments, the service management apparatus may illustratively have the following functions:

(1) the service management device sorts the storage service units according to the storage performance parameters, can exclude the storage service with the storage utilization rate exceeding a threshold value, and selects the candidate service unit information to provide for the file transmission service unit. Optionally, the service management apparatus may also form a second candidate storage service list according to the selected candidate storage service unit, so as to update the local first candidate storage service list of the file transfer service unit.

(2) The service management device receives a candidate storage service distribution request of a file transmission service unit; for example, the file transfer service is triggered by the candidate storage service in the local list not being sufficient or the local list not being updated too long.

(3) The service management device receives the storage service abnormity notice of the file transmission service unit, marks the abnormal storage service unit, cancels the distribution of the abnormally marked storage service unit when responding to the candidate storage service acquisition request of the file transmission service unit next time, and can not be selected as the candidate storage service unit. Possibly, the method can also carry out alarm processing and manual processing, and cancel the abnormal mark after the normal recovery is confirmed. At ordinary times, the storage service units with abnormal marks and not recovered can be checked, and the possibility that the storage service units are used as candidate storage service units is eliminated;

(4) if the number of available storage service units is greater than the number of candidate storage units of the preset allocation number, the service management device may extract the file transmission rate according to the task execution success notification of the file transmission service, perform weighted calculation according to the storage utilization rate calculated according to the acquired storage usage information to obtain the storage performance parameter, and select the preset allocation number with the higher storage performance parameter as the candidate storage service unit to allocate to the file transmission service initiating the request. If the available storage service units are less than the preset distribution quantity, directly taking the storage service units as candidate storage service units for distribution;

(5) the service management device receives a file exception notification of the file transmission service unit (triggered by the transmission failure times of the files exceeding a first preset threshold value), and performs alarm processing;

(6) the service management device receives the task execution success notification of the file transmission service unit and records the task execution success notification for calculating the storage performance parameters; or, performance evaluation of other objects, such as performance evaluation of file transfer service units, to illustratively load balance, etc.

It should be noted that the above functions are only shown by way of example, and may be modified according to actual needs, and the present invention is not limited thereto.

As shown in fig. 3, a schematic flow chart of the file transmission method in the embodiment of the present application is shown.

For example, the file transfer method may be applied to a file transfer service unit to transfer a video file to a storage service unit assigned by a service management apparatus.

The file transmission method comprises the following steps:

step S301: and the file transmission service unit puts the transmitted files into a file transmission queue according to the file transmission task.

In some embodiments, transmitted files with file sizes exceeding a preset capacity threshold may be excluded from the file transmission queue; wherein the preset capacity threshold is associated with a maximum capacity of a storage service unit. For example, if the size of a video file exceeds the upper limit of the storage service unit, the transmission will not succeed and it will need to be dequeued.

Step S302: the file transmission service unit determines a target storage service unit for storing the transmitted file according to the candidate storage service list.

Reference may be made to the generation principle of the candidate storage service list in the storage service allocation method in the embodiment of fig. 2, which is not repeated herein.

In some embodiments, the file transfer service may determine the destination storage service by matching the file size of the video file to the available capacity of the storage service. In the list, the storage service units may be arranged from high to low according to the storage performance parameter to indicate the allocation priority, and then the first storage service unit satisfying the size of the video file in the list is preferentially determined as the target storage service unit when the matching is performed. In an optional example, a storage service unit with relatively poor reliability can be excluded as a target storage service unit by using file transmission failure as a reference, for example, the storage service unit with failure occurring in the previous n times in the candidate storage service list is prevented from being selected as the target storage service unit, where n is greater than or equal to 1; specifically, for example, when a storage service unit in which the previous file upload failed is matched in the list, the non-selection is skipped.

And the file transmission service unit selects a target storage service unit according to the available storage service units in the local candidate storage service list. However, considering that the shortage will eventually occur with the use of the storage service unit, and it is necessary to request the service management apparatus to replenish the new storage service unit, in some examples, the following steps may be performed before step S302: determining whether the candidate storage service list meets a preset condition; wherein the preset conditions include: the available storage service units in the candidate storage service list are sufficient, and can be judged by that the number n of the storage service units in the list is lower than a preset distribution number threshold (for example, n <1 or n <2), or can also be judged by whether the capacity of the remaining candidate storage services is sufficient to meet the current video file transmission or file transmission task; and/or the non-updated time of the candidate storage service list exceeds a preset time threshold (indicating expiration and unreliability) so as to take a next action.

It can be understood that the preset condition may be constructed by whether the storage service unit is sufficient for individual construction, by whether the candidate storage service list is expired, or by whether the storage service unit is sufficient and the candidate storage service list is expired.

Taking the preset condition as an example that the storage service unit is sufficient and the candidate storage service list is not expired, step S302 may include:

in response to the preset condition being met, namely the storage service units are enough and the candidate storage service list is not expired, the file transmission service unit determines the storage service unit matched with the transmitted file in the candidate storage service list as a target storage service unit. With respect to the matching, for example, capacity matching, i.e., the capacity of the destination storage service unit is sufficient to store the transmitted video files; however, if the capacity of each storage service unit is sufficient, the storage service unit can be randomly selected as a target storage service unit; alternatively, the selection may be performed in order of high to low memory performance parameters.

Or, in response to the preset condition not being met (i.e. the storage service unit is insufficient, or the candidate storage service list is expired), the file transfer service unit sends a candidate storage service allocation request to the service management apparatus to request allocation of the storage service unit; and determining a target storage service unit by the matching mode based on the candidate storage service list updated corresponding to the allocation.

Step S303: and the file transmission service unit completes the file transmission task by using the target storage service unit.

Step S304: the file transfer service unit sends a transfer task execution success notification to the service management device in response to completion of the file transfer task.

Referring to the embodiment of fig. 2, the task execution success notification may include transmission performance parameters as a basis for calculating storage performance parameters, and the storage performance parameters as a basis for determining assignable storage service units. In possible examples, the task execution succeeds in notifying information of the recordable destination storage service unit, the size of the video file or data transmitted, the transmission time period, and the like. Alternatively, in some embodiments, the task execution success notification may also directly include the calculated file transfer rate, and may further include an average file transfer rate of each destination storage service unit in the task. For example, the task execution success notification includes: file name, file size, destination storage service unit, time spent uploading, average rate of uploading, etc.

When the file transmission fails, the failure information can be recorded for determining an abnormal file and an abnormal storage service unit. In a possible example, the recorded failure information may include: file name, file size, destination storage service, failure information, failure time, etc. of the video file that failed to be transmitted.

In some embodiments, the file transfer service unit may perform an exception handling action based on the statistical result of the transfer failure event to eliminate unreliable video files or storage service units, thereby avoiding congestion caused by continuous transfer failure due to an exception. The performing of the exception handling action based on the statistical result of the transmission failure event may include at least one of:

1) and in response to the transmission failure times of the files exceeding a first preset time threshold, the file transmission service unit removes the files from the file transmission queue, adds the files to the file transmission failure queue, and sends a file transmission failure notice to the service management device.

Specifically, reference may be made to the file transmission failure queue in fig. 1, which is used as a shunt queue of abnormal files in the file transmission queue, so as to avoid transmission congestion of the file transmission queue caused by the abnormal files. The anomaly alarm may be referenced by the service management means,

2) in response to the number of times of failure of continuous file transmission, which has occurred in the candidate storage service unit, exceeding a second preset number threshold, the file transmission service unit moves the candidate storage service unit out of the candidate storage service list, and sends a storage service exception notification to the service management device.

Specifically, if the number of consecutive transmission failures is too large, the storage service unit may have an abnormality, so the candidate storage service list may be excluded and the abnormality may be notified to the service management apparatus. Alternatively, referring to the embodiment of fig. 2, the service management device may mark an exception to the storage service unit corresponding to the storage service exception notification, without participating in allocation; and when the normal state is recovered, the allocation is carried out again.

Referring to fig. 4, a flow chart of a file transmission method in an application example of the present application is shown.

In fig. 4, the process includes:

step S401: calculating the time interval E from the last time of obtaining the storage service unit to the current time;

step S402: judging E > T2 or the number n <1 of storage service units, namely judging whether no enough storage service units are available or whether the time of the candidate storage service list not updated exceeds a threshold value; if yes, go to step S403; if not, the step S404 is entered;

step S403: sending a candidate storage service allocation request to a service management device;

step S404: judging whether the file size is abnormal, namely exceeding a preset capacity threshold; if yes, eliminating the large files exceeding the preset capacity threshold value from the file transmission queue, and reporting the large files abnormally; if not, go to step S405;

step S405: according to the file transmission task, putting the transmitted file into a file transmission queue;

step S406: matching a target storage service unit for the transmitted file in the candidate storage service list; the method comprises the following steps that a transmitted file is prevented from being allocated to a storage service unit which has failed in the last file transmission, and other storage service units can be randomly allocated to the transmitted file;

step S407: executing a file transmission task according to the matched target storage service unit; wherein, the transmission result of each file can be recorded;

step S408: and when the file is transmitted successfully, namely the task is completed, sending a task execution success notice to the service management device, and then starting the next file transmission task. Wherein the task execution success notification may include: file name, file size, destination storage service unit, upload time, upload average rate, etc.; this information can be obtained from the results of the previous transmission of each file.

Step S409: when there is a file transfer failure, the failure information may be recorded. Wherein the recording information includes: the file name, file size, destination storage service unit, failure information, failure time, etc. of the transmission failure may be returned to step S407, and then an attempt may be made to repeat the retransmission of the file of which the transmission failed.

Step S410: judging whether the transmission failure times of the transmitted files exceed a first preset time threshold value X or not;

if yes, go to step S411: removing the file from the file uploading queue, and notifying the service management device of the file transmission failure;

if not, the process proceeds to step S412: judging whether the continuous transmission failure times of the layer storage service unit exceed a second preset time threshold value Y or not;

if yes, the process goes to step S413, where the storage service unit is removed from the candidate storage list, and a storage service exception notification is sent to the service management apparatus; and can wait for the next file transmission task;

if not, step S414 is entered, and optionally, if the candidate storage list is sorted according to, for example, the storage performance parameter from high to low to preferentially select the storage performance parameter with higher storage performance parameter for file transmission, in step S414, the position of the candidate storage service unit may be adjusted to be at the last sorted position.

Referring to fig. 5, an interaction diagram of a file transfer process in an application example of the present application is shown.

Fig. 5 shows a file transfer service unit, a service management device, a storage service unit #1, a storage service unit #2, and a storage service unit # 3.

See the storage capacity collection box, which indicates that the service management device collects storage use information for the storage service unit #1, the storage service unit #2, and the storage service unit #3, respectively, see the storage capacity collection request and the storage capacity collection response in each pair. "loop" means a loop execution, for example, here the collection of the usage information of the storage capacity is periodically performed by a period T1.

In the lower block, the file transfer service unit periodically determines whether the number of storage service units is insufficient (less than 2), or is greater than a preset time threshold T2 (i.e., expires) from the last acquisition time interval (i.e., the non-updated time of the candidate storage service list), and if so, requests the service management device to "acquire the candidate storage service unit", i.e., requests the acquisition of the candidate storage service unit.

In the next lower two blocks, it is indicated that "file upload processing" is performed, which is divided into two cases of "file upload success" and "file upload failure", where file upload success in this example indicates that each file is uploaded successfully, i.e., the file transmission task is completed successfully, and it can be seen that, by matching each file of the file transmission task, i.e., file #1, file #2,. the file # n, in the candidate storage service list, for example, matching capacity and excluding the previous upload failure record to determine the destination storage service unit; or skipping the storage service unit which has the uploading failure record before and randomly distributing the rest units. After matching, the files #1, #2, and.

If there is a file upload failure, the file #2 upload target storage service unit, i.e., storage #2 failure, is schematically shown in the figure. If the file uploading failure times exceed X, the X is a first preset time threshold; if the number of the files exceeds X, the file transmission service unit can delete the files from the file transmission queue and report a file transmission failure notice to the service management device to prompt that the files are abnormal. If the storage service fails to upload for Y consecutive times on the storage service, Y is a second preset time threshold; if Y is exceeded, the file transfer service unit can move the storage service unit out of the candidate storage service list, such as storage #2, and report a storage service exception notification to the service management device to prompt storage #2 exception. The service management means may further mark it as abnormal without the parameters of the next allocation.

Fig. 6 is a block diagram of a document transportation device according to an embodiment of the present disclosure. The implementation of the file transfer device can refer to the file transfer method of the file transfer service unit in the previous embodiment, and therefore, the same technical features are not repeated in this embodiment.

The file transfer apparatus 600 includes:

a transmission task execution module 601, configured to place a file to be transmitted into a file transmission queue according to a file transmission task;

a destination storage determining module 602, configured to determine a destination storage service unit for storing the transferred file according to the candidate storage service list; wherein the candidate storage service list comprises a plurality of storage service units; each storage service unit is determined from each available storage service unit according to the storage performance parameters of each storage service unit by the service management device by using a preset storage service distribution rule and is provided to the file transmission service unit;

the transmission task execution module 601 is configured to complete a file transmission task by using a destination storage service unit;

an execution result reporting module 603, configured to send a transmission task execution success notification to the service management device in response to completion of the file transmission task, where the transmission task execution success notification includes a transmission performance parameter of the corresponding destination storage service unit, and is used as a reference for calculating the storage performance parameter when the storage service unit is determined again.

In some embodiments, the destination storage determination module 602 includes:

the condition determining module is used for determining whether the candidate storage service list meets a preset condition; wherein the preset conditions include: the storage service units available in the candidate storage service list are sufficient; and/or the non-updated time of the candidate storage service list exceeds a preset time threshold;

the first determining module is used for determining a storage service unit matched with the transmitted file in the candidate storage service list as a target storage service unit in response to the preset condition being met;

the storage service application module is used for responding to the situation that the preset condition is not met and sending a candidate storage service distribution request to the service management device to request the distribution of the storage service units; and a second determining module for determining a destination storage service unit based on the updated candidate storage service list corresponding to the allocation.

In some embodiments, the destination storage determination module 602 includes: and the avoidance module is used for avoiding selecting the storage service unit which has failed in the previous n times in the candidate storage service list as a target storage service unit, wherein n is more than or equal to 1.

In some embodiments, the file transfer device 600 further includes: and the exception handling module is used for executing exception handling actions based on the statistical result of the transmission failure event.

In some embodiments, the exception handling module includes at least one of:

1) the file exception handling module is used for removing the file from the file transmission queue in response to the fact that the transmission failure times of the file exceed a first preset time threshold, adding the file into the file transmission failure queue, and sending a file transmission failure notice to the service management device;

2) and the storage service exception handling module is used for responding to the condition that the failure times of continuous file transmission which have occurred to the candidate storage service unit exceed a second preset time threshold, moving the candidate storage service unit out of the candidate storage service list, and sending a storage service exception notification to the service management device.

In some embodiments, the file transfer device 600 includes: the file exclusion module is used for excluding the transmitted files with the file size exceeding a preset capacity threshold value from the file transmission queue; wherein the preset capacity threshold is associated with a maximum capacity of a storage service unit.

Fig. 7 is a block diagram of a storage service allocation apparatus according to an embodiment of the present application. The implementation of the storage service allocation apparatus may refer to the storage service allocation method of the service management apparatus in the previous embodiment, and therefore, the same technical features are not repeated in this embodiment.

The storage service allocating apparatus 700 includes:

a storage information acquisition module 701, configured to acquire storage use information of each storage service unit and calculate a capacity performance parameter;

an information extraction module 702, configured to extract the transmission performance parameters of the corresponding destination storage service unit according to the transmission task execution success notification received from the file transmission service unit regarding the completion of the file transmission task;

a storage performance calculating module 703, configured to calculate a storage performance parameter based on the transmission performance parameter and the capacity performance parameter corresponding to each storage service unit;

an information generating module 704, configured to determine candidate storage service units from the available storage service units based on the storage performance parameters of the storage service units by using a preset allocation rule;

a sending module 705, configured to send, in response to the candidate service allocation request, information of the determined storage service unit to the file transfer service unit, so as to determine a destination storage service unit for storing the transferred file in the file transfer task.

In some embodiments, the storage performance calculating module 703 is configured to calculate a weighted result of the transmission performance parameter and the capacity performance parameter.

In some embodiments, the information generating module 704 selects a preset allocation number of storage service units with a higher storage performance parameter for determining the storage service units therefrom.

In some embodiments, the information generating module 704 further includes: the first excluding module is used for excluding the storage service units with the capacity performance parameters exceeding the preset utilization rate threshold value from allocation.

In some embodiments, the storage service allocation apparatus 700 comprises: and the allocation module is used for responding to a candidate storage service allocation request of the file transmission service unit, determining a storage service unit and responding to the candidate storage service allocation request.

In some embodiments, the information generating module 704 is further configured to, in response to the number of available storage service units being lower than a preset allocation number, determine each of the available storage service units as the candidate storage service unit.

In some embodiments, the storage service allocation apparatus 700 comprises: an exception handling module, configured to, in response to a storage exception notification indicating that a storage service unit is abnormal, mark the corresponding storage service unit as abnormal by the service management apparatus to be restricted from being allocated; or, in response to the storage service unit indicating the exception returning to normal, removing the flag for the exception.

It should be noted that, all or part of the functional modules in the embodiments of fig. 6 and 7 may be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of program instruction products. The program instruction product includes one or more program instructions. The processes or functions according to the present application occur in whole or in part when program instruction instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The program instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.

In addition, the apparatuses disclosed in the embodiments of fig. 6 and fig. 7 can be implemented by other module division methods. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the modules described is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or modules may be combined or may be dynamic to another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or modules, and may be in an electrical or other form.

In addition, each functional module and sub-module in the embodiments of fig. 6 and 7 may be dynamically in one processing unit, or each module may exist alone physically, or two or more modules may be dynamically in one unit. The dynamic component can be realized in a form of hardware or a form of a software functional module. The dynamic components described above, if implemented in the form of software functional modules and sold or used as a stand-alone product, may also be stored in a computer readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

It should be noted that the flow or method representations represented by the flow diagrams of the above-described embodiments of the present application may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes other implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

For example, the order of the steps in the embodiments of fig. 2, fig. 3, fig. 4, etc. may be changed in a specific scenario, and is not limited to the above representation.

Fig. 8 is a schematic diagram of a circuit structure of a computer device according to an embodiment of the present application.

In some embodiments, the computer device 800 may be implemented in, for example, a server bank, a desktop, etc.

The computer device 800 includes a bus 801, a processor 802, a memory 803, and a communicator 804. The processor 802 and the memory 803 may communicate with each other via a bus 801. The memory 803 may have stored therein program instructions. The processor 802 implements a file transfer service unit or a service management apparatus in the embodiment of the present application by executing program instructions in the memory 803 to execute the steps in the file transfer method or the storage service allocation method.

The bus 801 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. Although only one thick line is shown in fig. 1 for ease of illustration, only one bus or type of bus is not shown.

In some embodiments, the processor 802 may be implemented as a Central Processing Unit (CPU), a micro-Processing Unit (MCU), a System On a Chip (System On Chip), or a field programmable logic array (FPGA). The Memory 803 may include a Volatile Memory (Volatile Memory) for temporary storage of data when the program is executed, such as a Random Access Memory (RAM).

The Memory 803 may also include a non-volatile Memory (non-volatile Memory) for data storage, such as a Read-Only Memory (ROM), a flash Memory, a Hard Disk Drive (HDD) or a Solid-State Disk (SSD).

The communicator 804 is used for communicating with the outside. In particular examples, the communicator 804 can include one or more wired and/or wireless communication circuit modules. For example, the wired communication circuit module may include one or more of a wired network card, a USB module, a serial interface module, and the like, for example. As another example, the wireless communication protocol followed by the wireless communication module includes: such as one or more of Near Field Communication (NFC) technology, Infrared (IR) technology, Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (Time-Division Code Division Multiple Access, TD-SCDMA), Long Term Evolution (LTE), BlueTooth (BlueTooth), Global Navigation Satellite System (GNSS), and the like.

A computer-readable storage medium may also be provided in an embodiment of the present application, and store program instructions, where the program instructions are executed to perform the file transmission method or the storage service allocation method in the previous embodiment.

That is, the method steps in the above-described embodiments are implemented as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium downloaded through a network, so that the method represented herein can be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA.

In summary, in the file transmission method, the storage service allocation method, the device, the apparatus, and the medium provided in the embodiments of the present application, the service device acquires and calculates capacity performance parameters of the storage service units, extracts corresponding transmission performance parameters according to the notification of the storage service units, combines the acquired storage performance parameters with accurate storage service unit performance, and determines each storage service unit according to the relevant preset storage service allocation rule to provide the storage service unit with the file transmission service unit, the file transmission service unit queries the storage service unit matching the transmitted file in the candidate storage service list, thereby completing the execution of the file transmission task, and can form a notification to feed back to the service device to form a closed loop, so that the effect of storage service allocation balance is continuously enhanced, the storage performance is optimized, and conditions such as file transmission failure and high availability guarantee can be used to eliminate unreliable storage service units, and the reliability of the storage service is improved.

It should be noted that, in the embodiments of the present application, some features need to be considered:

1) storage service consumer and storage service decoupling

The storage user can obtain the actually used storage service from the service management device only when the storage service is required to be used (insufficient or overdue), and the storage service is not forcibly distributed by the service management device before being used, so that the decoupling is realized, and the storage and the expansion are facilitated; and, the service management device can also perform scheduling and load balancing control of the storage service to optimize storage performance.

2) Storage scheduling and load balancing control taking into account file transfer rates

The method not only considers the balance of the storage utilization rate of the storage capacity, but also refers to the respective file transmission rates of the target storage units used by the file transmission service unit for transmitting the files recently during scheduling, and preferentially selects the candidate storage service unit with the higher transmission speed while ensuring the balanced use of the capacity.

3) Multi-candidate store and adjust exit mechanism

One of the main purposes of providing multiple storage service units is to ensure high availability of the system, and in the embodiment of the present application, when a storage unit is allocated, a manner of allocating multiple candidate storage service units is used to ensure high availability, and an adjustment exit mechanism of the candidate storage service units is provided, and the characteristics that multiple threads can be concurrently uploaded to multiple storage service units during file transfer are maintained. The storage performance is improved as much as possible while ensuring high availability.

The above embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.

Claims

1. A file transmission method is characterized in that a file transmission service unit transmits a video file to a storage service unit distributed by a service management device; the method comprises the following steps:

the file transmission service unit puts the transmitted file into a file transmission queue according to the file transmission task;

the file transmission service unit determines a target storage service unit for storing the transmitted file according to the candidate storage service list; wherein the candidate storage service list comprises a plurality of storage service units; each storage service unit is determined from each available storage service unit according to the storage performance parameters of each storage service unit by the service management device by using a preset storage service distribution rule and is provided to the file transmission service unit;

the file transmission service unit completes a file transmission task by using the target storage service unit;

and the file transmission service unit responds to the completion of the file transmission task and sends a transmission task execution success notice to the service management device, wherein the transmission task execution success notice comprises the transmission performance parameters of the corresponding target storage service unit and is used as a reference basis for calculating the storage performance parameters when the storage service unit is determined again.

2. The file transmission method according to claim 1, wherein the file transmission service unit determines a destination storage service unit for storing the transmitted file according to the candidate storage service list, comprising:

determining whether the candidate storage service list meets a preset condition; wherein the preset conditions include: the storage service units available in the candidate storage service list are sufficient; and/or the non-updated time of the candidate storage service list exceeds a preset time threshold;

in response to the preset condition being met, the file transmission service unit determines a storage service unit matched with the transmitted file in the candidate storage service list as a target storage service unit;

in response to the preset condition not being satisfied, the file transfer service unit sending a candidate storage service allocation request to the service management apparatus to request allocation of the storage service unit; and determining a destination storage service unit based on the candidate storage service list updated corresponding to the allocation.

3. The file transmission method according to claim 1, wherein the file transmission service unit determines a destination storage service unit for storing the transmitted file according to the candidate storage service list, comprising:

and avoiding selecting the storage service unit which has failed in the previous n times in the candidate storage service list as a target storage service unit, wherein n is more than or equal to 1.

4. The file transfer method according to claim 1, further comprising: an exception handling action is performed based on the statistics of the transmission failure event.

5. The file transmission method according to claim 4, wherein the performing of the exception handling action based on the statistical result of the transmission failure event comprises at least one of:

6. The file transfer method according to claim 1, comprising:

excluding the transmitted files with the file size exceeding a preset capacity threshold value from the file transmission queue; wherein the preset capacity threshold is associated with a maximum capacity of a storage service unit.

7. A storage service distribution method is characterized in that the method is applied to distributing storage service units for file transmission service units to transmit files; the method comprises the following steps:

the service management device collects the storage use information of each storage service unit and calculates capacity performance parameters;

the service management device extracts the transmission performance parameters of the corresponding target storage service unit according to the transmission task execution success notice which is received from the file transmission service unit and is about the completion of the file transmission task; the target storage service unit is used for storing the video files after transmission is finished;

the service management device calculates storage performance parameters based on the transmission performance parameters and the capacity performance parameters corresponding to each storage service unit;

the service management device determines candidate storage service units from all available storage service units based on the storage performance parameters of all the storage service units by using a preset allocation rule;

the service management apparatus transmits information of the determined storage service unit to the file transfer service unit in response to the candidate service allocation request.

8. The storage service allocation method according to claim 7, wherein the service management device calculates the storage performance parameter based on the transmission performance parameter and the capacity performance parameter corresponding to each storage service unit, and comprises:

and calculating the weighting result of the transmission performance parameter and the capacity performance parameter.

9. The storage service allocation method according to claim 7, wherein the service management device determines the storage service units based on the storage performance parameters of the storage service units by using a preset allocation rule, and the method comprises:

the service management device selects a preset distribution number of storage service units with higher storage performance parameters to be used for determining the storage service units.

10. The storage service allocation method according to claim 7 or 9, wherein the service management apparatus determines the storage service units based on the storage performance parameters of the storage service units by using a preset allocation rule, further comprising:

storage service units whose capacity performance parameters exceed a preset utilization threshold are excluded from allocation.

11. The storage service allocation method according to claim 7, comprising:

in response to a candidate storage service allocation request for a file transfer service unit, the service management apparatus determines a storage service unit in response to the candidate storage service allocation request.

12. The storage service allocation method according to claim 11, wherein the candidate storage service allocation request is triggered by a preset condition not being satisfied, the preset condition comprising: the candidate storage service in the candidate storage service list of the file transmission service unit is enough to complete the file transmission task; and/or the non-update time of the candidate storage service list exceeds a preset time threshold.

13. The storage service allocation method according to claim 7, comprising: in response to the number of available storage service units being less than a preset allocation number, the service management device takes each of the available storage service units as the candidate storage service unit.

14. The storage service allocation method according to claim 7, comprising: in response to a storage exception notification indicating that a storage service unit is abnormal, the service management apparatus marks the corresponding storage service unit as abnormal to be restricted from being allocated; alternatively, the flag of the exception is removed in response to the storage service unit indicating the exception returning to normal.

15. A file transmission device is characterized in that a file transmission service unit transmits a video file to a storage service unit distributed by a service management device; the device comprises:

the transmission task execution module is used for putting the transmitted file into a file transmission queue according to the file transmission task;

the destination storage determining module is used for determining a destination storage service unit for storing the transmitted file according to the candidate storage service list; wherein the candidate storage service list comprises a plurality of storage service units; each storage service unit is determined from each available storage service unit according to the storage performance parameters of each storage service unit by the service management device by using a preset storage service distribution rule and is provided to the file transmission service unit;

the transmission task execution module is used for completing a file transmission task by using the target storage service unit;

and the execution result reporting module is used for responding to the completion of the file transmission task and sending a transmission task execution success notice to the service management device, wherein the transmission task execution success notice comprises the transmission performance parameters of the corresponding target storage service unit and is used as a reference basis for calculating the storage performance parameters when the storage service unit is determined again.

16. A storage service distribution device is characterized in that the storage service distribution device is used for distributing storage service units for file transmission service units; the device comprises:

the storage information acquisition module is used for acquiring the storage use information of each storage service unit and calculating capacity performance parameters;

the information extraction module is used for extracting the transmission performance parameters of the corresponding target storage service unit according to the transmission task execution success notice which is received from the file transmission service unit and is about the completion of the file transmission task;

the storage performance calculation module is used for calculating storage performance parameters based on the transmission performance parameters and the capacity performance parameters corresponding to each storage service unit;

the information generation module is used for determining candidate storage service units from all available storage service units based on the storage performance parameters of all the storage service units by utilizing a preset distribution rule;

and the sending module is used for responding to the candidate service allocation request, and sending the information of the determined storage service unit to the file transmission service unit so as to determine a target storage service unit for storing the transmitted file in the file transmission task.

17. A computer device, comprising: a communicator, a memory, and a processor; the communicator is used for communicating with the outside; the memory is to store program instructions; the processor is configured to execute the program instructions to perform the file transfer method according to any one of claims 1 to 6; or, performing the storage service allocation method of any of claims 8 to 14.

18. A computer-readable storage medium, characterized in that program instructions are stored which, when executed, perform the file transfer method according to any one of claims 1 to 6; or, performing the storage service allocation method of any of claims 8 to 14.