CN110086850B

CN110086850B - File processing method and video network disk system

Info

Publication number: CN110086850B
Application number: CN201910210114.1A
Authority: CN
Inventors: 高�浩; 牛永会; 崔彬; 王春虎
Original assignee: Visionvera Information Technology Co Ltd
Current assignee: Visionvera Information Technology Co Ltd
Priority date: 2019-03-19
Filing date: 2019-03-19
Publication date: 2022-11-15
Anticipated expiration: 2039-03-19
Also published as: CN110086850A

Abstract

The embodiment of the invention provides a file processing method and a video network disk system, wherein the method comprises the following steps: the method comprises the steps that a network disk server receives a first target file which is uploaded by a network disk client side in parallel, and judges whether fragment identification and subfile numbers exist in the first target file or not; if the fragment identifier and the subfile number exist, acquiring data content in the first target file, and inserting the first target file into a file queue corresponding to the fragment identifier according to the subfile number; and finally, combining the files in the file queue according to the sequence and storing the combined files. The method can receive the numbered fragmented files uploaded in parallel, is beneficial to reducing the uploading duration of the files, and improves the reliability and the user experience.

Description

File processing method and video network disk system

Technical Field

The invention relates to the technical field of video networking, in particular to a file processing method and a video networking disk system.

Background

The video network disk is a storage service system in the video network, and in the process of executing an operation task, the operation task takes too long, for example, the uploading or downloading task of a file takes too long, which results in poor transmission reliability and poor user experience, so that the problems of too long operation task time and unreliable transmission of the video network disk need to be solved.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are proposed to provide a file processing method and a corresponding video network disk system that overcome or at least partially solve the above problems.

In order to solve the above problem, an embodiment of the present invention discloses a file processing method, which is applied to a video network disk system, where the disk system includes a disk client and a disk server, and the method includes:

the network disk server receives a first target file which is uploaded by the network disk client side in parallel;

the network disk server judges whether the first target file has a fragment identifier and a subfile number;

if the fragment identifier and the subfile number exist, the network disk server acquires data content in the first target file, and inserts the first target file into a file queue corresponding to the fragment identifier according to the subfile number;

if the fragment identifier exists but no subfile number exists, temporarily storing the first target file, and inserting the first target file into the discontinuous subfile numbers in the file queue when the receiving of the file queue meets a preset condition;

and the network disk server combines the files in the file queue according to the sequence and stores the files.

Optionally, the method further comprises:

the network disk client receives an operation instruction aiming at a second target file;

under the condition that the operation instruction is an uploading instruction, the network disk client detects the number of bytes contained in the second target file;

when the byte number is larger than a second byte number threshold value, the network disk client splits the target file into subfiles containing the second byte number, and adds fragment identifications and subfile numbers to the subfiles;

and the network disk client side uploads the subfiles to the network disk server in parallel through a video networking protocol.

Optionally, the step of inserting the first target file into the file queue between discontinuous subfile numbers when the receiving of the file queue meets a preset condition includes:

after the file is inserted into the file queue for the preset time period last time, acquiring discontinuous subfile numbers in the file queue;

and if the sub-file numbers have a file number difference, inserting the first target file into the discontinuous sub-file numbers.

Optionally, the method further comprises:

if the sub-file numbers have a plurality of file numbers, sequencing the plurality of temporarily stored first target files according to the initial content and the end content of the file content;

inserting the ordered plurality of first target files between the discontinuous subfile numbers.

Optionally, the step of sorting the plurality of first target files according to the start content and the end content of the file content includes:

aiming at two first target files, connecting the starting content of one first target file to the end content of the other first target file to obtain a candidate statement;

performing semantic analysis on the candidate sentences;

in case the semantic analysis is successful, determining that the one of the first target files is ordered after the other first target file.

The embodiment of the invention also discloses a network disk system of the video network, the network disk system comprises a network disk client and a network disk server, and the network disk server comprises:

the receiving module is used for receiving a first target file which is uploaded by the network disk client side in parallel;

the judging module is used for judging whether the first target file has a fragment identifier and a subfile number;

the first inserting module is used for acquiring data content in the first target file if a fragment identifier and a subfile number exist, and inserting the first target file into a file queue corresponding to the fragment identifier according to the subfile number;

the second inserting module is used for temporarily storing the first target file if fragment identification exists but no subfile number exists, and inserting the first target file into the file queue among discontinuous subfile numbers when the receiving of the file queue meets a preset condition;

and the merging module is used for merging the files in the file queue according to the sequence and storing the merged files.

Optionally, the network disk client includes:

the operation instruction receiving module is used for receiving an operation instruction aiming at the second target file;

the detection module is used for detecting the number of bytes contained in the second target file under the condition that the operation instruction is an uploading instruction;

the splitting module is used for splitting the target file into subfiles containing second byte numbers when the byte number is larger than a second byte number threshold value, and adding fragment identifications and subfile numbers to the subfiles;

and the uploading module is used for uploading the subfiles to the network disk server in parallel through a video networking protocol.

Optionally, the second plug-in module of the network disk server includes:

the subfile number obtaining sub-module is used for obtaining discontinuous subfile numbers in the file queue after the file is inserted into the file queue for the preset time period last time;

and the second inserting sub-module is used for inserting the first target file into the discontinuous subfile numbers if the subfile numbers have a file number difference.

Optionally, the second plug-in module of the network disk server further includes:

the sequencing submodule is used for sequencing the plurality of temporarily stored first target files according to the initial content and the ending content of the file content if the difference between the subfile numbers is a plurality of file numbers;

and the third inserting sub-module is used for inserting the plurality of ordered first target files into the discontinuous sub-file numbers.

Optionally, the sorting submodule of the second insertion module includes:

the candidate sentence determining unit is used for connecting the starting content of one first target file to the ending content of the other first target file to obtain a candidate sentence aiming at two first target files;

a semantic analysis unit, configured to perform semantic analysis on the candidate sentence;

a ranking determining unit, configured to determine that the one of the first target files is ranked after the other first target file if the semantic analysis is successful.

The embodiment of the invention has the following advantages:

according to the file processing method and the video network disk system provided by the embodiment of the invention, a disk server receives a first target file which is uploaded by a network disk client side in parallel, and judges whether the first target file has a fragment identifier and a subfile number; if the fragment identifier and the subfile number exist, acquiring data content in the first target file, and inserting the first target file into a file queue corresponding to the fragment identifier according to the subfile number; and finally, combining the files in the file queue according to the sequence and storing the combined files. The method can receive the numbered fragmented files which are uploaded in parallel, is beneficial to reducing the uploading duration of the files, and improves the reliability and the user experience.

Drawings

FIG. 1 is a schematic networking diagram of a video network of the present invention;

FIG. 2 is a schematic diagram of a hardware architecture of a node server according to the present invention;

fig. 3 is a schematic diagram of a hardware structure of an access switch of the present invention;

fig. 4 is a schematic diagram of a hardware structure of an ethernet protocol conversion gateway according to the present invention;

FIG. 5 is a flowchart of the steps of one embodiment of a method of file processing of the present invention;

FIG. 6 is a flowchart of the steps of another document processing method embodiment of the present invention;

FIG. 7 is a block diagram of a network disk server of a video network according to the present invention;

FIG. 8 is a block diagram of a network disk client of a video network according to the present invention;

fig. 9 is a block diagram of a video network disk system according to the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

The video networking is an important milestone of network development, is a real-time network, can realize high-definition video real-time transmission, and pushes a plurality of Internet applications to high-definition video, and high definition is face-to-face.

The video networking adopts a real-time high-definition video exchange technology, can integrate required services such as dozens of services of video, voice, pictures, characters, communication, data and the like on a system platform on a network platform, such as high-definition video conference, video monitoring, intelligent monitoring analysis, emergency command, digital broadcast television, delayed television, network teaching, live broadcast, VOD on demand, television mail, personal Video Recorder (PVR), intranet (self-office) channels, intelligent video broadcast control, information distribution and the like, and realizes high-definition quality video broadcast through a television or a computer.

To better understand the embodiments of the present invention, the following description refers to the internet of view:

some of the techniques applied by the video network are as follows:

network Technology (Network Technology)

Network technology innovation in video networking has improved over traditional Ethernet (Ethernet) to face the potentially enormous video traffic on the network. Unlike pure network Packet Switching (Packet Switching) or network Circuit Switching (Circuit Switching), the Packet Switching is adopted by the technology of the video networking to meet the Streaming requirement. The video networking technology has the advantages of flexibility, simplicity and low price of packet switching, and simultaneously has the quality and safety guarantee of circuit switching, thereby realizing the seamless connection of the whole network switching type virtual circuit and the data format.

Switching Technology (Switching Technology)

The video network adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the defects of the Ethernet on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, and directly bears an IP data packet. The user data does not require any format conversion across the entire network. The video networking is a higher-level form of the Ethernet, is a real-time exchange platform, can realize the real-time transmission of the whole-network large-scale high-definition video which cannot be realized by the existing Internet, and pushes a plurality of network video applications to high-definition and unification.

Server Technology (Server Technology)

The server technology on the video networking and unified video platform is different from the traditional server, the streaming media transmission of the video networking and unified video platform is established on the basis of connection orientation, the data processing capacity of the video networking and unified video platform is independent of flow and communication time, and a single network layer can contain signaling and data transmission. For voice and video services, the complexity of video networking and unified video platform streaming media processing is much simpler than that of data processing, and the efficiency is greatly improved by more than one hundred times compared with that of a traditional server.

Storage Technology (Storage Technology)

The super-high speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the media content with super-large capacity and super-large flow, the program information in the server instruction is mapped to the specific hard disk space, the media content is not passed through the server any more, and is directly sent to the user terminal instantly, and the general waiting time of the user is less than 0.2 second. The optimized sector distribution greatly reduces the mechanical motion of the magnetic head track seeking of the hard disk, the resource consumption only accounts for 20% of that of the IP internet of the same grade, but concurrent flow which is 3 times larger than that of the traditional hard disk array is generated, and the comprehensive efficiency is improved by more than 10 times.

Network Security Technology (Network Security Technology)

The structural design of the video network completely eliminates the network security problem troubling the internet structurally by the modes of independent service permission control each time, complete isolation of equipment and user data and the like, generally does not need antivirus programs and firewalls, avoids the attack of hackers and viruses, and provides a structural carefree security network for users.

Service Innovation Technology (Service Innovation Technology)

The unified video platform integrates services and transmission, and is not only automatically connected once whether a single user, a private network user or a network aggregate. The user terminal, the set-top box or the PC are directly connected to the unified video platform to obtain various multimedia video services in various forms. The unified video platform adopts a menu type configuration table mode to replace the traditional complex application programming, can realize complex application by using very few codes, and realizes infinite new service innovation.

Networking of the video network is as follows:

the video network is a centralized control network structure, and the network can be a tree network, a star network, a ring network and the like, but on the basis of the centralized control node, the whole network is controlled by the centralized control node in the network.

As shown in fig. 1, the video network is divided into an access network and a metropolitan network.

The devices of the access network part can be mainly classified into 3 types: node server, access switch, terminal (including various set-top boxes, coding boards, memories, etc.). The node server is connected to an access switch, which may be connected to a plurality of terminals and may be connected to an ethernet network.

The node server is a node which plays a centralized control function in the access network and can control the access switch and the terminal. The node server can be directly connected with the access switch or directly connected with the terminal.

Similarly, devices of the metropolitan network portion may also be classified into 3 types: a metropolitan area server, a node switch and a node server. The metro server is connected to a node switch, which may be connected to a plurality of node servers.

The node server is a node server of the access network part, namely the node server belongs to both the access network part and the metropolitan area network part.

The metropolitan area server is a node which plays a centralized control function in the metropolitan area network and can control a node switch and a node server. The metropolitan area server can be directly connected with the node switch or directly connected with the node server.

Therefore, the whole video network is a network structure with layered centralized control, and the network controlled by the node server and the metropolitan area server can be in various structures such as tree, star and ring.

The access network part can form a unified video platform (the part in the dotted circle), and a plurality of unified video platforms can form a video network; each unified video platform may be interconnected via metropolitan area and wide area video networking.

Visio networking device classification

1.1 devices in the video network of the embodiment of the present invention can be mainly classified into 3 types: servers, switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.). The video network as a whole can be divided into a metropolitan area network (or national network, global network, etc.) and an access network.

1.2 wherein the devices of the access network part can be mainly classified into 3 types: node servers, access switches (including ethernet gateways), terminals (including various set-top boxes, code boards, memories, etc.).

The specific hardware structure of each access network device is as follows:

a node server:

as shown in fig. 2, the network interface module 201, the switching engine module 202, the CPU module 203, and the disk array module 204 are mainly included;

the packets coming from the network interface module 201, the cpu module 203 and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of looking up the address table 205 on the incoming packet, thereby obtaining the direction information of the packet; and stores the packet in a queue of the corresponding packet buffer 206 based on the packet's steering information; if the queue of the packet buffer 206 is nearly full, it is discarded; the switching engine module 202 polls all packet buffer queues for forwarding if the following conditions are met: 1) The port send buffer is not full; 2) The queue packet counter is greater than zero. The disk array module 204 mainly implements control over the hard disk, including initialization, read-write, and other operations on the hard disk; the CPU module 203 is mainly responsible for protocol processing with an access switch and a terminal (not shown in the figure), configuring an address table 205 (including a downlink protocol packet address table, an uplink protocol packet address table, and a data packet address table), and configuring the disk array module 204.

The access switch:

as shown in fig. 3, the network interface module mainly includes a network interface module (a downlink network interface module 301 and an uplink network interface module 302), a switching engine module 303 and a CPU module 304;

wherein, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the Destination Address (DA), the Source Address (SA), the packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id) and enters the switching engine module 303, otherwise, discards the stream identifier; the packet (downstream data) coming from the upstream network interface module 302 enters the switching engine module 303; the data packet coming from the CPU module 204 enters the switching engine module 303; the switching engine module 303 performs an operation of looking up the address table 306 on the incoming packet, thereby obtaining the direction information of the packet; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in association with the stream-id; if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downlink network interface to the uplink network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the guiding information of the packet; if the queue of the packet buffer 307 is close to full, it is discarded.

The switching engine module 303 polls all packet buffer queues, which in this embodiment of the present invention is divided into two cases:

if the queue is from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queued packet counter is greater than zero; 3) Obtaining a token generated by a code rate control module;

if the queue is not from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) The port send buffer is not full; 2) The queue packet counter is greater than zero.

The rate control module 208 is configured by the CPU module 204, and generates tokens for packet buffer queues from all downstream network interfaces to upstream network interfaces at programmable intervals to control the rate of upstream forwarding.

The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306, and configuration of the code rate control module 308.

Ethernet protocol conversion gateway：

As shown in fig. 4, the apparatus mainly includes a network interface module (a downlink network interface module 401 and an uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a rate control module 408, an address table 406, a packet buffer 407, a MAC adding module 409, and a MAC deleting module 410.

Wherein, the data packet coming from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the ethernet MAC DA, the ethernet MAC SA, the ethernet length or frame type, the video network destination address DA, the video network source address SA, the video network packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id); then, the MAC deletion module 410 subtracts MAC DA, MAC SA, length or frame type (2 byte) and enters the corresponding receiving buffer, otherwise, discards it;

the downlink network interface module 401 detects the sending buffer of the port, and if there is a packet, obtains the ethernet MAC DA of the corresponding terminal according to the destination address DA of the packet, adds the ethernet MAC DA of the terminal, the MAC SA of the ethernet protocol gateway, and the ethernet length or frame type, and sends the packet.

The other modules in the ethernet protocol gateway function similarly to the access switch.

A terminal:

the system mainly comprises a network interface module, a service processing module and a CPU module; for example, the set-top box mainly comprises a network interface module, a video and audio coding and decoding engine module and a CPU module; the coding board mainly comprises a network interface module, a video and audio coding engine module and a CPU module; the memory mainly comprises a network interface module, a CPU module and a disk array module.

1.3 devices of the metropolitan area network part can be mainly classified into 2 types: node server, node exchanger, metropolitan area server. The node switch mainly comprises a network interface module, a switching engine module and a CPU module; the metropolitan area server mainly comprises a network interface module, a switching engine module and a CPU module.

2. Video networking packet definition

2.1 Access network packet definition

The data packet of the access network mainly comprises the following parts: destination Address (DA), source Address (SA), reserved byte, payload (PDU), CRC.

As shown in the following table, the data packet of the access network mainly includes the following parts:

DA

SA

Reserved

Payload

CRC

wherein:

the Destination Address (DA) is composed of 8 bytes (byte), the first byte represents the type of the data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are 256 possibilities at most, the second byte to the sixth byte are metropolitan area network addresses, and the seventh byte and the eighth byte are access network addresses;

the Source Address (SA) is also composed of 8 bytes (byte), and is defined to be the same as the Destination Address (DA);

the reserved byte consists of 2 bytes;

the payload part has different lengths according to the types of different datagrams, 64 bytes if it is a packet of various protocols, 32+1024=1056 bytes if it is a packet of unicast data, and certainly not limited to the above 2 types;

the CRC consists of 4 bytes and is calculated in accordance with the standard ethernet CRC algorithm.

2.2 packet definition for metropolitan area networks

The topology of a metropolitan area network is a graph and there may be 2, or even more than 2, connections between two devices, i.e., there may be more than 2 connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the metro network address of the metro network device is unique, and in order to accurately describe the connection relationship between the metro network devices, parameters are introduced in the embodiment of the present invention: a label to uniquely describe a metropolitan area network device.

In the present specification, the definition of the Label is similar to that of the Label of MPLS (Multi-Protocol Label Switch), and assuming that there are two connections between the device a and the device B, there are 2 labels for the packet from the device a to the device B, and there are 2 labels for the packet from the device B to the device a. The label is classified into an incoming label and an outgoing label, and assuming that the label (incoming label) of the packet entering the device a is 0x0000, the label (outgoing label) of the packet leaving the device a may become 0x0001. The network access process of the metro network is a network access process under centralized control, that is, address allocation and label allocation of the metro network are both dominated by the metro server, and the node switch and the node server are both passively executed, which is different from label allocation of MPLS, and label allocation of MPLS is a result of mutual negotiation between the switch and the server.

As shown in the following table, the data packet of the metro network mainly includes the following parts:

DA

SA

Reserved

label (R)

Payload

CRC

Namely Destination Address (DA), source Address (SA), reserved byte (Reserved), tag, payload (PDU), CRC. The format of the tag may be defined by reference to the following: the tag is 32 bits with the upper 16 bits reserved and only the lower 16 bits used, and its position is between the reserved bytes and payload of the packet.

Based on the above characteristics of the video network, one of the core concepts of the embodiments of the present invention is proposed, in which, following a protocol of the video network, a network disk client uploads a first target file to a network disk server in parallel, and the network disk server receives and processes the first target file.

Referring to fig. 5, a flowchart illustrating steps of an embodiment of a file processing method according to the present invention is shown, where the method may be applied to a video network disk system, where the disk system includes a disk client and a disk server, and specifically may include the following steps:

step 501, the network disk server receives a first target file uploaded by the network disk client side in parallel.

In the embodiment of the invention, the network disk system of the video network is a storage service system in the video network and comprises a network disk client and a network disk server. When a user performs task operation on the network disk client, the network disk client sends the task to the network disk server for processing. When the network disk client executes an instruction of uploading a local file of a user to the video network disk, the network disk server receives a first target file which is uploaded by the user through the network disk client in parallel.

Parallel processing is a computational method in which two or more processes can be performed simultaneously in a computer system. To use parallel processing, a program needs to be parallelized first, that is, parts of work are allocated to different processing processes or threads. In the embodiment of the invention, the network disk client distributes the uploading tasks of the plurality of fragments to different processing processes for processing, and the parallel processing mainly aims to save the uploading time of large files. Moreover, if the chip is disconnected in the middle, the uploading can be continued from a certain failed chip.

Step 502, the network disk server determines whether the first target file has a fragment identifier and a subfile number.

In the embodiment of the invention, if the network disk client judges that the first target file uploaded by the user occupies a large space, the first target file is split into a plurality of fragments to be sent. Each fragment may include a video networking frame header, a video networking protocol packet header, and a payload portion of a packet. Each slice may be set to contain a set number of bytes, for example, for a packet in the 8F95 protocol format for video networking, a packet of the protocol may carry 1480 bytes, and then each slice may be set to 1480 bytes. Of course, if the remaining number of bytes is less than 1480, the last slice may include only the remaining number of bytes, or add null characters to make up.

The fragment identifier is used for marking the name of a target file to which the fragment data packet belongs, and the subfile number is used for marking the position information of the fragment in the target file.

If the first target file does not comprise the fragment identification and the subfile number information, the network disk server directly responds to the uploading instruction; if the first target file comprises the fragment identifier and the subfile number information, the network disk server needs to process the first target file.

Step 503, if there is a fragment identifier and a subfile number, the network disk server obtains the data content in the first target file, and inserts the first target file into the file queue corresponding to the fragment identifier according to the subfile number.

In the embodiment of the present invention, if there is a fragment identifier and a subfile number, it indicates that the first target file is a subfile in a large file, and it can be determined which large file the first target file belongs to according to the fragment identifier. And when the first target file is specifically merged, inserting the first target file into other subfile queues with the same fragment identifier. The insertion position is determined based on the subfile number.

Step 504, if the fragment identifier exists but no subfile number exists, temporarily storing the first target file, and inserting the first target file into the file queue between discontinuous subfile numbers when the reception of the file queue meets a preset condition.

In the embodiment of the present invention, if there is a fragment identifier but there is no subfile number, the insertion location of the first target file cannot be determined, the first target file may be temporarily stored in a cache, and when the reception of the file queue meets a preset condition, the first target file is inserted between discontinuous subfile numbers in the file queue.

In a possible embodiment of the present invention, the step of inserting the first target file between discontinuous subfile numbers in the file queue when the receiving of the file queue meets a preset condition includes the following sub-steps 5041 and 5042:

a substep 5041, obtaining discontinuous subfile numbers in the file queue after the file is inserted into the file queue for the preset time period last time;

in sub-step 5042, if there is a file number difference between the subfile numbers, inserting the first target file between the discontinuous subfile numbers.

In the embodiment of the present invention, the preset time period may be preset according to actual conditions, for example, a transmission time may be estimated according to a transmission speed of the subfiles, or a longer upper limit transmission time may be set, and after the preset time period, the first target file is inserted between discontinuous subfile numbers. Wherein, the discontinuous subfile numbers differ by one file number. For example, the subfile number of the first target file is ctx2, it should be inserted between the discontinuous subfile numbers ctx1 and ctx 3.

In one possible embodiment of the present invention, step 504 may further include sub-steps 5043 and 5044 of:

sub-step 5043, if there are a plurality of file numbers between the subfile numbers, sorting the plurality of first target files according to the initial content and the end content of the file content;

sub-step 5044, inserting the sorted first target files between the discontinuous subfile numbers.

In the embodiment of the present invention, if there is a difference between the subfile numbers by a plurality of file numbers, it indicates that the subfile after the sorting arrives at the network disk server before the first target files, at this time, the first target files may be sorted, and the sorting rule may be that the file content of each first target file is analyzed, and the sorting is performed according to the start content and the end content of the file content. And after the first target files are successfully sorted, inserting the sorted first target files into discontinuous subfile numbers.

In one possible embodiment of the present invention, sub-step 5043, may include the following sub-steps A5-A7:

substep A5: aiming at two first target files, connecting the starting content of one first target file to the end content of the other first target file to obtain a candidate statement;

substep A6: performing semantic analysis on the candidate sentences;

substep A7: in case the semantic analysis is successful, determining that the one of the first target files is ordered after the other first target file.

In the embodiment of the invention, the first target files are combined pairwise, a candidate sentence is obtained after the initial content of the next first target file in the combination is connected with the end content of the previous first target file, the semantic logic, the grammar correctness and other aspects of the candidate sentence are analyzed, if the semantic logic and the grammar are correct, the combination is successful, and the next first target file is sequenced behind the previous first target file. After the sorting is completed, the step of inserting the first target file may be performed.

In one possible embodiment of the invention, under the condition of failure of semantic analysis, the connection sequence of the two first target files is exchanged, and then the semantic analysis is carried out; and when the semantic analysis of the two connection sequences fails, performing content connection and semantic analysis on the next group of first target files according to the steps A5-A7.

In the embodiment of the present invention, if the semantic logic or the syntax correctness of the candidate sentence is not correct, it indicates that the combination fails, at this time, the connection order of the two first target files needs to be exchanged, a new candidate sentence is obtained after the exchange, the semantic logic and the syntax correctness are verified again for the new candidate sentence, and if the verification is successful, it indicates that the new connection order is correct, the insertion step of the first target file is performed according to the connection order.

And if the semantic analysis of the two connection sequences fails, combining the first target files pairwise again, and performing content connection and semantic analysis on the new candidate sentences formed by combination until all the semantic analysis of the first target files is successful.

According to the embodiment of the invention, when the difference between the subfile numbers is multiple file numbers, the temporarily stored multiple first target files are sequenced according to the initial content and the end content of the file content, and the files are spliced through semantic analysis, so that the uploading success rate of the first target files is improved.

And 505, combining the files in the file queue according to the sequence by the network disk server, and storing.

In the embodiment of the present invention, the specific combining manner may be: when a fragment is uploaded, the network disk server stores the fragment and returns a unique identifier ctx to the network disk client, wherein the identifier ctx is associated with the current fragment; the network disk client stores the identification of each fragment and the sequence of the identification; the network disk server provides an interface, the network disk client requests the fragments to be combined by the interface, the identifiers are sequentially informed to the network disk server when the network disk client requests the fragments to be combined, ctx0, ctx1 and ctx2 … …, and the server finds the corresponding fragments according to the sequence of the identifiers and combines the fragments into a complete file.

In summary, in the embodiment of the present invention, a first target file uploaded by a network disk client in parallel is received by a network disk server, and whether the first target file has a fragment identifier and a subfile number is determined; if the fragment identifier and the subfile number exist, acquiring data content in the first target file, and inserting the first target file into a file queue corresponding to the fragment identifier according to the subfile number; and finally, combining the files in the file queue according to the sequence and storing the combined files. The method can receive the numbered fragmented files which are uploaded in parallel, is beneficial to reducing the uploading duration of the files, and improves the reliability and the user experience.

Referring to fig. 6, a flowchart illustrating steps of another embodiment of a file processing method according to the present invention is shown, where the file processing method may be executed before step 501 in fig. 5, and the file processing method specifically includes the following steps:

step 601, the network disk client receives an operation instruction for a second target file.

The operation instruction may be an operation of uploading the second target file stored locally to the internet of things disk by the user, or may be a downloading operation of the second target file stored in the internet of things disk.

Step 602, in a case that the operation instruction is an upload instruction, the network disk client detects a number of bytes included in the second target file.

And under the condition that the operation instruction is an instruction for uploading the second target file to the network disk, the network disk client detects the number of bytes contained in the second target file. If the byte number is smaller, directly responding to an uploading instruction, and uploading the second target file to the network disk server; if the byte number is larger, in order to ensure the operation efficiency of the uploading task and reduce the task execution time, the network disk client side processes the second target file.

Step 603, when the byte number is greater than the second byte number threshold value, the network disk client splits the target file into subfiles containing the second byte number, and adds a fragment identifier and a subfile number to the subfiles.

In the embodiment of the present invention, the second byte number threshold may be preset according to a type of an adopted video networking protocol, the fragment identifier is used to label a name of a target file to which the fragment data packet belongs, and the subfile number is used to label location information of the fragment in the target file. Each fragment carries a fragment identifier and a subfile number, so that the receiving end can conveniently merge a plurality of subfiles.

And step 604, the network disk client side uploads the subfiles to the network disk server in parallel through a video networking protocol.

In an embodiment of the present invention, parallel processing is a computational method in which two or more processes can be performed simultaneously in a computer system. To use parallel processing, a program needs to be parallelized first, that is, parts of work are allocated to different processing processes (threads). In the embodiment of the invention, the uploading tasks of a plurality of fragments are distributed to different processing processes for processing, and the parallel processing mainly aims to save the uploading time of large files. And if the chip is disconnected in the middle, the uploading can be continued from a certain failed chip.

The video network comprises a plurality of transmission protocols, and the second target file in the embodiment of the invention needs to be fragmented and marked in the transmission process, so that the video network 8F95 protocol can be selected for sub-file uploading.

The specification of the 8F95 protocol may be as shown in table 1 below:

TABLE 1

It can be seen that the 8F95 protocol specification includes a source device video network MAC (i.e. a network disk client address) and a destination device video network MAC (i.e. a network disk server address), and a service start instruction is also marked in the header of the data packet, i.e. it indicates that the protocol used for this transmission is the 8F95 protocol. In addition, the unit W of the field length represents a Word (Word), and 16 bits are a Word, which represents the binary digit number of the computer processing instruction or data. In addition, in the specification of the 8F95 protocol, the packet sequence number and the total packet number are also included, which facilitates the network disk server to recombine and splice the received subfiles, so that the second target file is presented in the original sequence, and the integrity of data transmission is ensured.

In addition, the embodiment of the invention provides the method embodiment of uploading the target file from the network disk client to the network disk server, and for the method of downloading the target file from the network disk server to the network disk client, the uploading mode in the invention can be referred, and only the operations executed by the sending end and the receiving end are exchanged.

In summary, in the embodiment of the present invention, the network disk client receives an operation instruction for the second target file; under the condition that the operation instruction is an uploading instruction, the network disk client detects the number of bytes contained in the second target file; when the byte number is larger than a second byte number threshold value, the network disk client splits the target file into subfiles containing the second byte number, and adds fragment identifications and subfile numbers to the subfiles; and the network disk client uploads the subfiles to the network disk server in parallel through a video networking protocol. In the method, when the byte number of the second target file is larger than the corresponding threshold value, the network disk client splits the second target file, adds the identification and the serial number to the split subfiles so as to facilitate the network disk server to correctly combine the subfiles, and then uploads the plurality of subfiles in parallel, so that the uploading time of the large file is saved, and the problem of unreliable transmission is solved.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 7, a block diagram of a network disk server according to the present invention is shown, where the network disk server 700 may be applied in a video network disk system, and specifically includes the following modules:

a receiving module 701, configured to receive a first target file uploaded by the network disk client in parallel;

a determining module 702, configured to determine whether the first target file has a fragment identifier and a subfile number;

a first inserting module 703, configured to obtain data content in the first target file if there is a fragment identifier and a subfile number, and insert the first target file into a file queue corresponding to the fragment identifier according to the subfile number;

a second inserting module 704, configured to temporarily store the first target file if there is a fragment identifier but there is no subfile number, and insert the first target file into a discontinuous subfile number in the file queue when the reception of the file queue meets a preset condition;

and a merging module 705, configured to merge the files in the file queue in order and store the merged files.

The network disk server 700 may implement the steps in the file processing method shown in fig. 5, and the related description may refer to steps 501 to 505, which are not described herein again.

Optionally, the second plug-in module 704 of the network disk server includes:

and the second inserting sub-module is used for inserting the first target file into the discontinuous subfile numbers if the subfile numbers differ by one file number.

Optionally, the second inserting module 704 of the network disk server further includes:

the sequencing sub-module is used for sequencing the plurality of temporarily stored first target files according to the initial content and the end content of the file content if the difference between the subfile numbers is a plurality of file numbers;

Optionally, the sorting submodule of the second insertion module includes:

a semantic analysis unit, configured to perform semantic analysis on the candidate sentences;

Referring to fig. 8, a block diagram of a network disk client according to the present invention is shown, where the network disk client 800 may be applied in a video network disk system, and specifically may include the following modules:

an operation instruction receiving module 801, configured to receive an operation instruction for a second target file;

a detecting module 802, configured to detect, when the operation instruction is an upload instruction, a number of bytes included in the second target file;

a splitting module 803, configured to split the target file into subfiles including a second byte number when the byte number is greater than a second byte number threshold, and add a fragment identifier and a subfile number to the subfiles;

an uploading module 804, configured to upload the subfiles to the network disk server in parallel through a video networking protocol.

The network disk server 800 may implement the steps in the file processing method shown in fig. 6, and the relevant description may refer to steps 601 to 604, which are not described herein again.

Referring to fig. 9, a block diagram of a video network disk system according to the present invention is shown, and the video network disk system 900 may specifically include the disk server 700 in fig. 7 and the disk client 800 in fig. 8.

The modules included in the network disk server 700 and the network disk client 800 are respectively the same as the modules in fig. 7 and fig. 8, and are not described again in this embodiment of the present invention.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal device that comprises the element.

The above detailed description is given to a file processing method and a video network disk system provided by the present invention, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A file processing method is applied to a network disk system of a video network, wherein the network disk system comprises a network disk client and a network disk server, and the method comprises the following steps:

if the fragment identifier and the subfile number exist, the network disk server acquires the data content in the first target file, and inserts the first target file into a file queue corresponding to the fragment identifier according to the subfile number;

if the fragment identifier exists but no subfile number exists, temporarily storing the first target file, and inserting the first target file into the discontinuous subfile numbers in the file queue when the receiving of the file queue meets a preset condition; the preset condition comprises a preset time period after the first target file is detected to have no subfile number;

the network disk server combines and stores the files in the file queue according to the sequence;

wherein, when the receiving of the file queue meets a preset condition, the step of inserting the first target file between discontinuous subfile numbers in the file queue comprises:

if the subfile numbers differ by one file number, inserting the first target file into the discontinuous subfile numbers;

2. The method of claim 1, further comprising:

and the network disk client uploads the subfiles to the network disk server in parallel through a video networking protocol.

3. The method according to claim 1, wherein the step of sorting the plurality of first target files according to the starting content and the ending content of the file contents comprises:

performing semantic analysis on the candidate sentences;

4. A network disk system of a video network is characterized in that the network disk system comprises a network disk client and a network disk server, and the network disk server comprises:

the receiving module is used for receiving the first target file uploaded by the network disk client;

the merging module is used for merging the files in the file queue according to the sequence and storing the files;

the second insertion module of the network disk server comprises:

the second inserting sub-module is used for inserting the first target file into the discontinuous subfile numbers if the subfile numbers differ by one file number;

5. The system of claim 4, wherein the web client comprises:

6. The system of claim 4, wherein the ordering submodule of the second insertion module comprises: