CN109600410B

CN109600410B - Data storage system and method

Info

Publication number: CN109600410B
Application number: CN201710944091.8A
Authority: CN
Inventors: 许爱秋; 汪渭春; 林起芊
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2017-09-30
Filing date: 2017-09-30
Publication date: 2021-11-09
Anticipated expiration: 2037-09-30
Also published as: CN109600410A

Abstract

The invention relates to a data storage system and a data storage method, and belongs to the technical field of video monitoring. The internal network storage system and the external network storage system in the data storage system both adopt a management node-storage node architecture, and local storage volumes and FTP directories are arranged in the storage nodes, so that multimedia data of the FTP directories in the internal network storage nodes can be read through FTP clients running on a gatekeeper, and then the multimedia data are ferred to the FTP directories of the external network storage nodes.

Description

Data storage system and method

Technical Field

The invention relates to the technical field of video monitoring, in particular to a data storage system and a data storage method.

Background

In recent years, with the enhancement of public safety awareness, video monitoring systems have come into play. In order to ensure the safety of each place, the safety control department can adjust and watch the monitoring video data of each place at any time, and if necessary, the safety control department also needs to adjust and watch the real-time monitoring of each place so as to carry out unified scheduling and commanding. In the video monitoring system, the network structure is divided according to the property of the monitored place, and places such as important units, key departments and the like all belong to places in an internal network due to confidential information related to each department; the internal network and the external network respectively and independently work to ensure the security of confidential information in the internal network, a gatekeeper needs to be established between the internal network and the external network to realize physical isolation between the internal network and the external network, and multimedia data transmission must be carried out between the internal network and the external network across the gatekeeper in order to realize uniform calling and flexible transmission of multimedia data in the internal network and the external network.

In the related technology, independent FTP (File Transfer Protocol) servers are respectively deployed in an internal network and an external network, a snapshot machine in the internal network uploads multimedia data to an intranet FTP server and an intranet cloud storage system, and the multimedia data in the intranet cloud storage system is moved to the external network cloud storage system by a built-in application in a gateway, so that the cross-gateway multimedia data transmission between the internal network and the external network is realized.

In the process of implementing the invention, the inventor finds that the related art has at least the following problems:

in order to realize the transmission of multimedia data from the intranet to the extranet, independent FTP servers need to be deployed in the intranet and the extranet, when a snapshot machine uploads the multimedia data to the intranet FTP server, if the data volume of the multimedia data which needs to be uploaded currently is large, and the time requirement for synchronizing the multimedia data to the extranet FTP server by the intranet FTP server is short, large pressure can be generated on the FTP servers of the intranet and the extranet, at the moment, a plurality of FTP servers need to be deployed in the intranet and the extranet to meet the performance requirement for uploading the multimedia data, the deployment requirement on equipment in the intranet and the extranet is high, and the deployment process is complex.

Disclosure of Invention

To overcome the problems in the related art, the present invention provides a data storage system and method.

On one hand, the data storage system comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the first management node of the intranet storage system is used for receiving a resource allocation request of a snapshot machine and allocating a target first storage node in the plurality of first storage nodes to the snapshot machine;

the target first storage node is used for storing the multimedia data and the short UUID into a cloud storage volume of the target first storage node and an FTP (file transfer protocol) directory of the target first storage node when receiving the multimedia data and the short UUID sent by the snapshot machine;

the gateway is used for reading the multimedia data and the short UUID in the FTP directory of the target first storage node through the FTP client, and sending the multimedia data and the short UUID to the FTP directory of the target second storage node of the external network storage system;

and the target second storage node is used for storing the multimedia data and the short UUID into a cloud storage volume of the target second storage node when the multimedia data and the UUID sent by the gatekeeper are received in the FTP directory of the target second storage node.

On one hand, the method is applied to an intranet storage system of a data storage system, and the data storage system comprises the intranet storage system, a network gate and an extranet storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the method comprises the following steps:

a first management node of the intranet storage system receives a resource allocation request of a snapshot machine, and allocates a target first storage node in the plurality of first storage nodes to the snapshot machine;

when the target first storage node receives the multimedia data and the short universal unique identification UUID sent by the snapshot machine, the multimedia data and the short UUID are stored in a cloud storage volume of the target first storage node and an FTP directory of the target first storage node, and the multimedia data and the short UUID in the FTP directory of the target first storage node are sent to the extranet storage system by an FTP client on the gateway.

On one hand, the method is applied to an external network storage system of a data storage system, and the data storage system comprises an internal network storage system, a network gate and the external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

any second storage node receives the multimedia data and the short UUID sent by the FTP client of the gatekeeper;

the second storage node stores the multimedia data and the short UUID into an FTP directory of the second storage node;

the second storage node stores the multimedia data and the short UUID in a cloud storage volume of the second storage node.

On one hand, the data storage method is applied to a data storage system, and the data storage system comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

when the target first storage node receives multimedia data and a short Universal Unique Identifier (UUID) sent by the snapshot machine, storing the multimedia data and the short UUID into a cloud storage volume of the target first storage node and an FTP (file transfer protocol) directory of the target first storage node;

the gateway reads the multimedia data and the short UUID in the FTP directory of the target first storage node through the FTP client, and sends the multimedia data and the short UUID to the FTP directory of the target second storage node of the external network storage system;

and when the target second storage node receives the multimedia data and the short UUID sent by the gatekeeper in the FTP directory of the target second storage node, storing the multimedia data and the short UUID into a cloud storage volume of the target second storage node.

the first management node of the intranet storage system is used for managing storage resources of the plurality of first storage nodes, and each first storage node is used for storing data and the short UUID of the data in the cloud storage volume of the first storage node; or the like, or, alternatively,

and the second management node of the extranet storage system is used for managing the storage resources of the plurality of second storage nodes, and each second storage node is used for storing data and the short UUID of the data in the cloud storage volume and the FTP directory of the second storage node.

On one hand, the data query method is provided, and comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the method comprises the steps that multimedia data and a short UUID of the multimedia data are stored in a cloud storage volume of a target first storage node; the first management node receives a multimedia data query request of a terminal, wherein the multimedia data query request carries a first virtual address; redirecting the multimedia data query request to a target first storage node indicated by the first virtual address according to the first virtual address; the target first storage node receives the multimedia data query request and returns the multimedia data indicated by the first virtual address to the terminal; or the like, or, alternatively,

storing multimedia data and a short UUID (user identifier) of the multimedia data in a cloud storage volume and an FTP (file transfer protocol) directory of a target second storage node; the second management node receives a multimedia data query request of a terminal, wherein the multimedia data query request carries a second virtual address; redirecting the multimedia data query request to a target second storage node indicated by the second virtual address according to the second virtual address; and the target second storage node receives the multimedia data query request and returns the multimedia data indicated by the second virtual address to the terminal.

The technical scheme provided by the embodiment of the invention can have the following beneficial effects:

the internal network storage system and the external network storage system in the data storage system adopt a management node-storage node architecture, and a local storage volume and an FTP directory are arranged in the storage nodes, so that multimedia data of the FTP directory in the internal network storage nodes can be read through an FTP client running on a gatekeeper, and then the multimedia data are ferred to the FTP directory of the external network storage nodes.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1A is an architecture diagram of a data storage system, shown in accordance with an exemplary embodiment;

FIG. 1B is a schematic diagram illustrating a data storage system in accordance with an exemplary embodiment;

FIG. 1C is a schematic diagram illustrating a multimedia data storage format in accordance with an exemplary embodiment;

FIG. 2 is a data flow diagram of a data storage system according to an embodiment of the present invention;

FIG. 3 is a flow chart of a data storage method according to an embodiment of the present invention;

FIG. 4 is a flowchart of a data query method provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a data storage device according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present invention;

FIG. 7 is a block diagram illustrating a computer device 700, according to an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

The video monitoring system is divided into an internal network and an external network, wherein the internal network comprises a snapshot machine, an internal network bayonet server, an internal network database and an internal network storage system; the external network storage system comprises an external network bayonet server, an external network database and an external network storage system, and the internal network storage system and the external network storage system form a data storage system in the video monitoring system. Referring to fig. 1A, both the intranet storage system and the extranet storage system adopt a distributed architecture and are divided into a management node and a plurality of storage nodes according to physical forms. The block device on each storage node is divided into two parts, one part is used as a cloud storage volume by a cloud storage service, and the other part is used as an FTP service by mounting a local file system, and is specifically provided as an FTP directory.

Specifically, referring to fig. 1B, the intranet storage system includes a first management node and a plurality of first storage nodes, where each first storage node includes a cloud storage volume and an FTP directory supporting a file transfer protocol FTP service; the extranet storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; the network gate runs with FTP client.

The functions of the intranet storage system, the gatekeeper and the extranet storage system are introduced based on the data storage process as follows:

an intranet storage system: first management node and a plurality of first storage nodes

The first management node is used for managing storage resources of the plurality of first storage nodes, and each first storage node is divided into a cloud storage volume and an FTP directory, wherein the FTP directory is an FTP service mounted with a local file system. Each first storage node corresponds to a unique first node identifier, and the first node identifiers of the first storage nodes are different.

The first management node of the intranet storage system is used for receiving a resource allocation request of the snapshot machine and allocating a target first storage node in the plurality of first storage nodes to the snapshot machine; the target first storage node is configured to store the multimedia data and the short UUID into the cloud storage volume of the target first storage node and the FTP directory of the target first storage node when receiving the multimedia data and the short Universally Unique Identifier UUID (Universally Unique Identifier) sent by the snapshot machine. It should be noted that the target first storage node can synchronously write the multimedia data and the short UUID into the FTP directory of the target first storage node.

The first management node is used for determining a target first storage node from the plurality of first storage nodes according to a preset rule, sending a first node identifier of the target first storage node to the snapshot machine, and writing multimedia data and the short UUID into the target first storage node by the snapshot machine. The first management node may determine, according to a load balancing principle, one second storage node whose current load meets a requirement from among the plurality of second storage nodes as a target first storage node, where of course, each first storage node may correspond to one hash value range, and the first management node may further calculate a hash value of the multimedia data based on a hash value algorithm, so as to select the first storage node in the hash value range corresponding to the hash value as the target first storage node.

When the storage node stores the multimedia data and the short UUID correspondingly, the short UUID can be stored in a data header of the multimedia data (as shown in fig. 1C), and the multimedia data carries the short UUID corresponding to the short UUID, so that when the multimedia data is subsequently transmitted to the extranet storage system, the UUID of the multimedia data remains unchanged, the same multimedia data can be directly determined in the intranet storage system and the extranet storage system according to the short UUID, the mapping relation of physical addresses of the upper-layer application storage multimedia data in the intranet storage system and the extranet storage system is omitted, and the phenomenon that when the upper-layer application is abnormal, the mapping relation is permanently lost, and the multimedia data cannot be searched in the intranet storage system and the extranet storage system is avoided.

After the multimedia data and the short UUID are stored in the cloud storage volume of the target first storage node, the multimedia data are stored in the storage nodes of the intranet, so that the multimedia data correspond to a physical address in the intranet, and the target first storage node stores the mapping relation between the physical address and the short UUID. The target first storage node may generate a mapping table as shown in table 1, and store the mapping between the short UUID and the physical address in the mapping table.

TABLE 1

Short UUID	abc
		Physical address	def
Time stamp information	22:02:06
		Random code	123456

In order to record the information of the multimedia data in more detail and enhance the confidentiality of the multimedia information, in practical applications, the time stamp information and the random code of the multimedia data can be added to the mapping relation table. It should be noted that, the manner of storing the mapping relationship between the short UUID and the physical address is not specifically limited in the present invention. Besides being stored in a database, the mapping relation between the physical address of the multimedia data and the short UUID is backed up in a storage volume at the bottom layer of the cloud storage system and is stored together with the multimedia data. Even if the mapping information of the local database is lost, the mapping information can be recovered through the tool, and the data security is improved.

It should be noted that, when the target first storage node stores the multimedia data and the short UUID in the FTP directory, on one hand, a multimedia data name may be generated for each multimedia data, where the multimedia data name includes the short UUID, and the multimedia data name are stored correspondingly; on the other hand, the multimedia data may be directly stored in correspondence with the short UUID, which is not specifically limited in the embodiment of the present invention.

In order to record the storage position of the multimedia data, so that when the multimedia data is queried in an intranet, the multimedia data can be directly acquired at the storage position, and the target first storage node can send the short UUID to the first management node; the first management node is further configured to return the short UUID and the first node identifier of the target first storage node to the snapshot machine, send a first virtual address to an intranet card oral server by the snapshot machine, and store the first virtual address to an intranet database by the intranet card port server, where the first virtual address is generated based on the short UUID and the first node identifier, so as to subsequently acquire multimedia data based on the first virtual address. The virtual address may be in the form of a URL (Uniform Resource Locator).

In addition, the snapshot machine in the intranet is used for acquiring multimedia data of the position where the snapshot machine is located, generating a short UUID of the multimedia data, storing the multimedia data and the short UUID to a target first storage node of the intranet storage system based on the target first storage node distributed by the first management node, and uploading a first virtual address to the intranet bayonet server when the first virtual address of the multimedia data is received; the multimedia data can be picture data or video data, each multimedia data corresponds to a unique short UUID, and the UUIDs of the multimedia data are different. The short UUID is used for identifying the multimedia data corresponding to the short UUID, and when the short UUID is generated for each multimedia data by the snapshot machine, the short UUID may be generated by using a preset Algorithm, where the preset Algorithm may be a SHA1(Secure Hash Algorithm ) digest Algorithm, a Boost (portable) library, or the like.

The intranet bayonet server is used for receiving the first virtual address transmitted by the snapshot machine, transmitting the first virtual address to the intranet database and storing the first virtual address by the intranet database; and after receiving the first virtual address transmitted by the intranet access point server, the intranet database stores the first virtual address.

Network gate

The gateway is used for reading the multimedia data and the short UUID in the FTP directory of the target first storage node through the FTP client, and sending the multimedia data and the short UUID to the FTP directory of the target second storage node of the external network storage system; the gatekeeper can periodically detect whether new multimedia data and a short UUID are stored in the FTP directory of each first storage node in the intranet storage system through the FTP client, and when detecting that the target first storage node stores the new multimedia data and the short UUID, the gatekeeper executes the processes of reading the multimedia data and the short UUID in the FTP directory of the target first storage node and sending the multimedia data and the short UUID to the extranet storage system.

The gatekeeper is also used for deleting the multimedia data and the short UUID stored in the FTP directory of the target first storage node after sending the multimedia data and the short UUID to the FTP directory of the target second storage node of the extranet storage system, so that the data volume stored in the intranet storage system can be reduced, and the pressure of the intranet storage system is relieved.

It should be noted that, because the FTP client in the gatekeeper may manage a plurality of intranet storage systems and extranet storage systems, in order to establish a corresponding relationship between the intranet FTP service and the extranet FTP service, and avoid a situation that a data transmission object error occurs when the multimedia data and the short UUID are transmitted, the first service address of the intranet FTP service and the second service address of the extranet FTP service are correspondingly stored in the FTP client. Therefore, before the gatekeeper works, a first service address of the intranet FTP service of the intranet storage system needs to be acquired, a second service address of the extranet FTP service of the extranet storage system needs to be acquired, the first service address and the second service address are correspondingly stored to the FTP client in the gatekeeper, so that when multimedia data and short UUIDs are subsequently transmitted to the extranet storage system, the extranet storage system corresponding to the intranet storage system can be determined, and errors of a transmission object are avoided.

Correspondingly, in the embodiment of the invention, since the corresponding relation between the first service address and the second service address is stored in the FTP client, after the FTP client acquires the multimedia data and the short UUID in the intranet FTP service, the FTP client can determine the second service address corresponding to the first service address according to the first service address of the intranet storage system, and transmit the multimedia data and the short UUID to the extranet storage system indicated by the second service address.

An extranet storage system: a second management node and a plurality of second storage nodes

The second management node is used for managing storage resources of a plurality of second storage nodes, and each second storage node is divided into a cloud storage volume and an FTP directory, wherein the FTP directory is an FTP service mounted with a local file system. Each second storage node corresponds to a unique second node identifier, and the second node identifiers of the second storage nodes are different.

For any second storage node, the second storage node is configured to store the multimedia data and the short UUID into the cloud storage volume of the second storage node when the multimedia data and the UUID sent by the gatekeeper are received in the FTP directory of the second storage node. It should be noted that, when the second storage node stores the multimedia data and the short UUID in the FTP directory, on one hand, a multimedia data name may be generated for each multimedia data, where the multimedia data name includes the short UUID, and the multimedia data name are stored correspondingly; on the other hand, the multimedia data can be directly stored in correspondence with the short UUID, so that if the multimedia data and the multimedia data name are stored in the FTP directory of the first storage node, the multimedia data and the multimedia data name are stored in correspondence; and if the multimedia data stored in the FTP directory of the first storage node is the short UUID, storing the multimedia data corresponding to the short UUID.

In the embodiment of the invention, the FTP directory of the second storage node firstly stores the multimedia data and the short UUID transmitted by the gatekeeper, so that the multimedia data and the short UUID are newly added in the external network storage system every time the multimedia data and the short UUID are newly added in the internal network storage system. For each second storage node in the external network storage system, the second storage node periodically detects whether the FTP directory stores the newly added multimedia data and the newly added short UUID, and when detecting that the FTP directory stores the newly added multimedia data and the newly added short UUID, the newly added multimedia data and the newly added short UUID are obtained, so that the newly added multimedia data and the newly added short UUID are stored in the cloud storage volume, and the transmission of the multimedia data and the newly added short UUID from the internal network storage system to the external network storage system is completed. After the multimedia data and the short UUID are stored in the extranet storage system, a mapping relationship similar to that in the intranet storage system can be formed, and details are not repeated here.

Further, the second storage node generates a second virtual address of the multimedia data according to the short UUID and a second node identifier of the second storage node, and sends the second virtual address to a second management node, the second management node sends the second virtual address to an extranet gate server, and the extranet gate server stores the second virtual address in an extranet database.

The framework provided by the embodiment of the invention provides the following beneficial effects:

the internal network storage system and the external network storage system in the data storage system both adopt a management node-storage node architecture, and local storage volumes and FTP directories are arranged in the storage nodes, so that multimedia data of the FTP directories in the internal network storage nodes can be read through FTP clients running on a gatekeeper, and then the multimedia data are ferred to the FTP directories of the external network storage nodes.

Further, the following advantageous effects are also provided:

(1) the multimedia data of the snapshot machine are directly stored in the cloud storage volume of the intranet storage system and the local FTP service, and can also be directly stored in the cloud storage volume of the extranet storage system and the local FTP service through the ferry of the network gate, so that a scheme for directly storing the multimedia data is provided. Because the data stream does not pass through the bayonet server, the performance pressure of the platform bayonet server is reduced.

(2) The high-speed ferry of the multimedia data between the internal network and the external network is realized by combining a gatekeeper program, after the multimedia data is ferred to the external network FTP service by the internal network FTP service, the updating of the multimedia data is detected by the built-in cloud storage program, the multimedia data is automatically written into the cloud storage volume, and therefore the automatic synchronization scheme of the multimedia data between two clouds across the gatekeeper is provided.

(3) Because the intranet storage system and the extranet storage system both adopt the short UUID for identification when storing the multimedia data, the upper application platform of the intranet and the extranet can adopt the same short UUID to access the same multimedia data in different cloud storage systems of the intranet and the extranet, and the step of recording the mapping relation by an external program is omitted.

For convenience of understanding, the data storage method provided in the embodiment of the present invention is described below based on the architecture of the data storage system, and may be described with reference to the data flow in fig. 2 and the specific step interaction in fig. 3:

301. and a first management node of the intranet storage system receives a resource allocation request of the snapshot machine and allocates a target first storage node in the plurality of first storage nodes to the snapshot machine.

The snapshot machine collects multimedia data at the position, and sends a resource allocation request to a first management node in the intranet storage system to request storage resources to store the multimedia data. Specifically, the first management node is configured to determine a target first storage node from the plurality of first storage nodes according to a preset rule, and send a first node identifier of the target first storage node to the snapshot machine. And when the snapshot machine acquires the first node identifier of the target first storage node, the multimedia data and the short UUID of the multimedia data are sent to the target first storage node.

302. The target first storage node receives the multimedia data and the short UUID sent by the snapshot machine, and stores the multimedia data and the short UUID into the cloud storage volume of the target first storage node and the FTP directory of the target first storage node.

In one possible implementation, the target first storage node sends the short UUID to the first management node; the first management node also returns the short UUID and the first node identification of the target first storage node to the snapshot machine; the snapshot machine sends a first virtual address to an intranet card oral server, the intranet card port server stores the first virtual address to an intranet database, and the first virtual address is generated based on the short UUID and the first node identifier.

303. And the gatekeeper determines that newly added multimedia data and short UUIDs exist in the target first storage node through periodic detection of the FTP client, and reads the multimedia data and the short UUIDs in the FTP directory of the target first storage node.

304. And the gatekeeper sends the multimedia data and the short UUID to an FTP directory of a target second storage node of the external network storage system.

Further, after the gatekeeper sends the multimedia data and the short UUID to the FTP directory of the target second storage node of the external network storage system, the multimedia data and the short UUID stored in the FTP directory of the target first storage node are deleted.

305. And when the target second storage node receives the multimedia data and the UUID sent by the gatekeeper in the FTP directory of the target second storage node, storing the multimedia data and the short UUID into the cloud storage volume of the target second storage node.

Further, the target second storage node generates a second virtual address of the multimedia data according to the short UUID and a second node identifier of the target second storage node, and sends the second virtual address to a second management node, and the second management node sends the second virtual address to an extranet bayonet server; and the external network card port server stores the second virtual address in an external network database.

According to the method provided by the embodiment of the invention, the FTP client running on the gatekeeper is used for reading the multimedia data of the FTP directory in the intranet storage node and ferrying the multimedia data to the FTP directory of the extranet storage node, and the ferrying process can ensure that the data transmission from the intranet to the extranet does not depend on independent FTP servers in the intranet and the extranet, so that the deployment is simpler and the requirements are lower. Further, the method provided by the embodiment of the invention makes full use of the advantages of the distributed cluster architecture of the video cloud storage system, and the FTP service is built in the storage nodes of the cloud storage system, so that the overhead of the FTP server is saved. Meanwhile, the FTP service is also deployed for the cluster, so that the network gate application can access the FTP service concurrently, and the performance of cross-network-gate image storage is improved.

The embodiment of the invention also provides a data storage system, which comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the extranet storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; the network gate runs with FTP client. The specific system structure may be the same as that shown in fig. 1A, fig. 1B, and fig. 2, and for the data storage system, when the multimedia data is stored, the data storage system may further provide a data query function for the user, and based on the data query function, the functions of the intranet storage system, the gatekeeper, and the extranet storage system are introduced:

taking an intranet storage system as an example, a first management node of the intranet storage system is used for managing storage resources of the plurality of first storage nodes, and each first storage node is used for storing data and a short UUID of the data in a cloud storage volume of the first storage node;

the first management node is used for receiving a multimedia data query request of a terminal, wherein the multimedia data query request carries a first virtual address; redirecting the multimedia data query request to a target first storage node indicated by the first virtual address according to the first virtual address; the target first storage node is used for receiving the multimedia data query request and returning the multimedia data indicated by the first virtual address to the terminal.

The first management node is used for analyzing the first virtual address to obtain the short UUID and a first node identifier; redirecting the multimedia data query request to a target first storage node indicated by the first node identification;

the target first storage node is used for receiving the multimedia data query request, acquiring the physical address of the multimedia data according to the short UUID, acquiring the multimedia data from the physical address, and returning the multimedia data to the terminal.

The terminal can access an intranet storage system through the platform and search an intranet database based on a certain retrieval condition, so that a first virtual address of multimedia data is obtained, the first virtual address is carried in a multimedia data query request and is sent to a first management node, the first management node can redirect the multimedia data query request of the platform to a storage node which actually stores the data through a URL (uniform resource locator) redirection method, the storage node obtains a physical address of the multimedia data through analysis, the multimedia data is obtained based on the physical address and is returned to the platform, and the multimedia data is returned to the terminal through the platform. Certainly, the redirecting may also refer to redirecting the multimedia data query request to a cloud storage service, so that the cloud storage service can analyze and determine a physical address, that is, an address of which node the data is stored at, and then acquire the multimedia data from the node and return the multimedia data to the platform.

Taking the extranet storage system as an example, the second management node of the extranet storage system is configured to manage storage resources of the plurality of second storage nodes, and each second storage node is configured to store data and a short UUID of the data in the cloud storage volume and the FTP directory of the second storage node.

The second management node is used for receiving a multimedia data query request of the terminal, wherein the multimedia data query request carries a second virtual address; redirecting the multimedia data query request to a second storage node indicated by the second virtual address according to the second virtual address;

the second storage node is used for receiving the multimedia data query request and returning the multimedia data indicated by the second virtual address to the terminal.

Further, the second management node is configured to parse the second virtual address to obtain the short UUID and a second node identifier; redirecting the multimedia data query request to a second storage node indicated by the second node identification;

the second storage node is used for receiving the multimedia data query request, acquiring the physical address of the multimedia data according to the short UUID, acquiring the multimedia data from the physical address, and returning the multimedia data to the terminal.

The terminal can access the extranet storage system through the platform and search the extranet database based on certain retrieval conditions, so that a second virtual address of the multimedia data is obtained, the second virtual address is carried in the multimedia data query request and is sent to the second management node, the second management node can redirect the multimedia data query request of the platform to a storage node which actually stores the data through a URL (uniform resource locator) redirection method, the storage node obtains a physical address of the multimedia data through analysis, the multimedia data obtained based on the physical address is returned to the platform, and the multimedia data is returned to the terminal through the platform. Certainly, the redirecting may also refer to redirecting the multimedia data query request to a cloud storage service, so that the cloud storage service can analyze and determine a physical address, that is, an address of which node the data is stored at, and then acquire the multimedia data from the node and return the multimedia data to the platform.

It should be noted that, because both the intranet storage system and the extranet storage system use the short UUID for identification when storing the multimedia data, when the user queries the multimedia data through their respective platforms, the same short UUID corresponding to the media data is the same, and the step of recording the mapping relationship by an external program is omitted.

Of course, the storage method of the multimedia data in the data storage system may be the same as the storage method in the data storage system, and may also use a different data flow method for storage, which is not limited in the embodiment of the present invention.

For convenience of understanding, the data storage method provided in the embodiment of the present invention is described below based on the architecture of the data storage system, and may be described by referring to the specific step interaction in fig. 4, taking the query of the intranet storage system as an example:

401. and the intranet platform receives the retrieval condition of the terminal and acquires the first virtual address meeting the retrieval condition from the intranet database according to the retrieval condition.

The intranet platform can be an intranet platform application program, for an intranet inquiry system, the intranet platform application program runs, and an inquiry process can be realized based on the program logic of the intranet platform application program.

402. And the intranet platform sends a multimedia data query request to the first management node, wherein the multimedia data query request carries the first virtual address.

403. When the first management node receives the first virtual address, the first virtual address is analyzed to obtain a short UUID and a first node identifier.

404. The first management node redirects the multimedia data query request.

It should be noted that after the first management node obtains the short UUID and the first node identifier, the first virtual address of the intranet platform may be redirected to the actually stored storage node, that is, the first management node may return the short UUID and the first node identifier to the intranet platform, and the intranet platform redirects based on the short UUID and the first node identifier, and sends the multimedia data query request to the redirected access address.

405. And the intranet platform sends the multimedia data query request to the target first storage node indicated by the first node identification based on the redirected multimedia data query request.

406. And the target first storage node receives the multimedia data query request, acquires the physical address of the multimedia data through the short UUID, and acquires the multimedia data through the physical address.

Since the storage node already establishes the mapping relationship between the short UUID and the physical address in the storage process, the physical address and thus the requested multimedia data can be acquired based on the mapping relationship.

The above-mentioned step 404-step 406 are the process of acquiring, by the intranet storage system, the multimedia data corresponding to the short UUID from the local storage volume of the target first storage node indicated by the first node identifier.

407. And the target first storage node sends the multimedia data to the intranet platform.

It should be noted that the processes shown in step 401 and step 407 are query processes of the intranet, and the query processes of the extranet are the same as the query processes of the intranet, and are not described herein again.

Fig. 5 is a schematic structural diagram of a data storage device according to an embodiment of the present invention. The device is applied to an intranet storage system of a data storage system, and the data storage system comprises the intranet storage system, a network gate and an extranet storage system; the data storage device comprises a management service module 501, a local storage service module 502 and an FTP service module 503, wherein an FTP client runs on the gateway;

the management service module is used for receiving a resource allocation request of the snapshot machine and allocating a target first storage node in the plurality of first storage nodes to the snapshot machine;

the local storage service module and the FTP service module respectively store the multimedia data and the short universal unique identifier UUID sent by the snapshot machine into a cloud storage volume and an FTP directory, and the FTP client on the gatekeeper sends the multimedia data and the short UUID in the FTP directory to the external network storage system.

In a possible implementation manner, the management service module is further configured to return the short UUID and the first node identifier of the target first storage node to the snapshot machine, send a first virtual address to an intranet card oral server by the snapshot machine, and store the first virtual address to an intranet database by the intranet card port server, where the first virtual address is generated based on the short UUID and the first node identifier.

In a possible implementation manner, the management service module is further configured to receive a multimedia data query request of the terminal, where the multimedia data query request carries the first virtual address; redirecting the multimedia data query request to a target first storage node indicated by the first virtual address according to the first virtual address;

the local storage service module is also used for receiving the multimedia data query request and returning the multimedia data indicated by the first virtual address to the terminal.

In a possible implementation manner, the management service module is further configured to parse the first virtual address to obtain the short UUID and the first node identifier; redirecting the multimedia data query request to a target first storage node indicated by the first node identification;

the local storage service module is also used for receiving the multimedia data query request, acquiring the physical address of the multimedia data according to the short UUID, acquiring the multimedia data from the physical address, and returning the multimedia data to the terminal.

Fig. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present invention. The device is applied to an external network storage system of a data storage system, wherein the data storage system comprises an internal network storage system, a network gate and an external network storage system; the data storage device comprises a management service module 601, a local storage service module 602 and an FTP service module 603, wherein an FTP client runs on the gateway;

the management service module 601 is used for managing the local storage service module 602 and the FTP service module 603;

the FTP service module 603 is used to receive the multimedia data and short UUID sent by the FTP client of the gatekeeper; the system is used for storing the multimedia data and the short UUID into an FTP directory;

the local storage service module 602 is configured to store the multimedia data and the short UUID in a cloud storage volume.

In one possible implementation, the local storage service module is configured to send the short UUID to the management service module;

the local storage service module 602 is configured to generate a second virtual address of the multimedia data according to the short UUID and a second node identifier of a current second storage node, send the second virtual address to the management service module 601, send the second virtual address to the extranet socket server by the management service module 601, and store the second virtual address in the extranet database by the extranet socket server.

In a possible implementation manner, the management service module 601 is further configured to receive a multimedia data query request of the terminal, where the multimedia data query request carries the second virtual address; redirecting the multimedia data query request to a second storage node indicated by the second virtual address according to the second virtual address;

the local storage service module is also used for receiving the multimedia data query request and returning the multimedia data indicated by the second virtual address to the terminal.

In a possible implementation manner, the management service module 601 is further configured to parse the second virtual address to obtain the short UUID and a second node identifier; redirecting the multimedia data query request to a second storage node indicated by the second node identification;

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

FIG. 7 is a block diagram illustrating a computer device 700, according to an example embodiment. For example, the computer device 700 may be provided as a server. Referring to fig. 7, a computer device 700 includes a processor 722, and memory resources, represented by memory 732, for storing instructions, e.g., application programs, that are executable by the processor 722. Memory 732 has stored therein computer-executable instructions that may be executed on the processor. In addition, the processor, when executing the computer executable instructions, implements the multimedia data storage method described above.

The computer device 700 may also include a power component 726 configured to perform power management of the computer device 700, and a wired or wireless network interface 750 configured to configure the computer device700 to a network, and an input-output (I/O) interface 757. The computer device 700 may operate based on an operating system, such as Windows Server, stored in memory 732^TM，Mac OS X^TM，Unix^TM,Linux^TM，FreeBSD^TMOr the like.

An embodiment of the present invention further provides a computer-readable storage medium, where when an instruction in the storage medium is executed by a processor of a card writing device, the card writing device is enabled to execute the above multimedia data storage method.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. A data storage system is characterized by comprising an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the target second storage node is used for storing the multimedia data and the short UUID into a cloud storage volume of the target second storage node when the multimedia data and the UUID sent by the gatekeeper are received in an FTP directory of the target second storage node;

the first management node is further used for returning the short UUID and a first node identifier of the target first storage node to the snapshot machine, the snapshot machine sends a first virtual address to an intranet card oral server, the intranet card port server stores the first virtual address to an intranet database, and the first virtual address is generated based on the short UUID and the first node identifier.

2. The system according to claim 1, wherein the first management node is configured to determine a target first storage node from the plurality of first storage nodes according to a preset rule, and send a first node identifier of the target first storage node to the snapshot machine.

3. The system of claim 1, wherein the target second storage node is further configured to generate a second virtual address of the multimedia data according to the short UUID and a second node identifier of the target second storage node, and send the second virtual address to a second management node, and the second management node is configured to send the second virtual address to an extranet portal server, and the extranet portal server stores the second virtual address in an extranet database.

4. The system of claim 1, wherein the gatekeeper is further configured to delete the multimedia data and the short UUID stored in the FTP directory of the target first storage node after sending the multimedia data and the short UUID to the FTP directory of the target second storage node of the extranet storage system.

5. The system according to claim 4, wherein the gatekeeper is configured to periodically detect, through the FTP client, whether new multimedia data and a short UUID are stored in the FTP directory of each first storage node in the intranet storage system, and when the multimedia data and the short UUID are detected, perform the processes of reading the multimedia data and the short UUID in the FTP directory of the target first storage node and sending the read multimedia data and the short UUID to the extranet storage system.

6. A multimedia data storage method is characterized in that the method is applied to an intranet storage system of a data storage system, and the data storage system comprises the intranet storage system, a network gate and an extranet storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the method comprises the following steps:

when the target first storage node receives the multimedia data and the short universal unique identification UUID sent by the snapshot machine, the multimedia data and the short UUID are stored in a cloud storage volume of the target first storage node and an FTP directory of the target first storage node, and the FTP client on the gatekeeper sends the multimedia data and the short UUID in the FTP directory of the target first storage node to the extranet storage system;

the method further comprises the following steps:

the first management node returns the short UUID and a first node identifier of the target first storage node to the snapshot machine, the snapshot machine sends a first virtual address to an intranet card oral server, the intranet card oral server stores the first virtual address to an intranet database, and the first virtual address is generated based on the short UUID and the first node identifier.

7. The method of claim 6, wherein assigning a target first storage node of the plurality of first storage nodes to the snapshot machine comprises:

and the first management node determines a target first storage node from the plurality of first storage nodes according to a preset rule, and sends a first node identifier of the target first storage node to the snapshot machine.

8. The method of claim 7, further comprising:

the first management node receives a multimedia data query request of a terminal, wherein the multimedia data query request carries a first virtual address; redirecting the multimedia data query request to a target first storage node indicated by the first virtual address according to the first virtual address;

and the target first storage node receives the multimedia data query request and returns the multimedia data indicated by the first virtual address to the terminal.

9. The method of claim 8,

the first management node analyzes the first virtual address to obtain the short UUID and a first node identifier; redirecting the multimedia data query request to a target first storage node indicated by the first node identification;

and the target first storage node receives the multimedia data query request, acquires a physical address of the multimedia data according to the short UUID, acquires the multimedia data from the physical address, and returns the multimedia data to the terminal.

10. A data storage method is characterized in that the method is applied to an external network storage system of a data storage system, wherein the data storage system comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the method comprises the following steps:

the second storage node stores the multimedia data and the short UUID in a cloud storage volume of the second storage node;

the method further comprises the following steps:

and the second storage node generates a second virtual address of the multimedia data according to the short UUID and a second node identifier of the second storage node, and sends the second virtual address to a second management node, the second management node sends the second virtual address to an external network port server, and the external network port server stores the second virtual address to an external network database.

11. The method according to claim 10, wherein the second management node receives a multimedia data query request of a terminal, wherein the multimedia data query request carries a second virtual address; redirecting the multimedia data query request to a second storage node indicated by the second virtual address according to the second virtual address;

and the second storage node receives the multimedia data query request and returns the multimedia data indicated by the second virtual address to the terminal.

12. The method of claim 11,

the second management node analyzes the second virtual address to obtain the short UUID and a second node identifier; redirecting the multimedia data query request to a second storage node indicated by the second node identification;

and the second storage node receives the multimedia data query request, acquires a physical address of the multimedia data according to the short UUID, acquires the multimedia data from the physical address, and returns the multimedia data to the terminal.

13. A data storage method is characterized in that the method is applied to a data storage system, wherein the data storage system comprises an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the method comprises the following steps:

when the target second storage node receives the multimedia data and the short UUID sent by the gatekeeper in the FTP directory of the target second storage node, the multimedia data and the short UUID are stored in a cloud storage volume of the target second storage node;

the method further comprises the following steps:

the first management node returns the short UUID and the first node identification of the target first storage node to the snapshot machine;

the snapshot machine sends a first virtual address to the intranet card oral server,

and the intranet bayonet server stores the first virtual address to an intranet database, and the first virtual address is generated based on the short UUID and the first node identifier.

14. The method according to claim 13, wherein the first management node determines a target first storage node from the plurality of first storage nodes according to a preset rule, and sends a first node identifier of the target first storage node to the snapshot machine.

15. The method of claim 13 wherein the gatekeeper deletes the multimedia data and the short UUID stored in the FTP directory of the target first storage node after sending the multimedia data and the short UUID to the FTP directory of the target second storage node of the extranet storage system.

16. The method of claim 13, wherein the target second storage node generates a second virtual address of the multimedia data according to the short UUID and a second node identifier of the target second storage node, and sends the second virtual address to a second management node, and the second management node sends the second virtual address to an extranet portal server;

and the external network card port server stores the second virtual address to an external network database.

17. A data storage system is characterized by comprising an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

the second management node of the extranet storage system is used for managing storage resources of the plurality of second storage nodes, and each second storage node is used for storing data and short UUIDs of the data in the cloud storage volume and FTP directory of the second storage node;

the first management node is used for receiving a multimedia data query request of a terminal, wherein the multimedia data query request carries a first virtual address; redirecting the multimedia data query request to a target first storage node indicated by the first virtual address according to the first virtual address;

the target first storage node is configured to receive the multimedia data query request, and return the multimedia data indicated by the first virtual address to the terminal.

18. The system of claim 17, wherein the first management node is configured to resolve the first virtual address to obtain the short UUID and a first node identifier; redirecting the multimedia data query request to a target first storage node indicated by the first node identification;

and the target first storage node is used for receiving the multimedia data query request, acquiring a physical address of the multimedia data according to the short UUID, acquiring the multimedia data from the physical address, and returning the multimedia data to the terminal.

19. The system according to claim 17, wherein the second management node is configured to receive a multimedia data query request of a terminal, where the multimedia data query request carries a second virtual address; redirecting the multimedia data query request to a target second storage node indicated by the second virtual address according to the second virtual address;

and the target second storage node is used for receiving the multimedia data query request and returning the multimedia data indicated by the second virtual address to the terminal.

20. The system of claim 19, wherein the second management node is configured to resolve the second virtual address to obtain the short UUID and a second node identifier; redirecting the multimedia data query request to a target second storage node indicated by the second node identification;

and the target second storage node is used for receiving the multimedia data query request, acquiring a physical address of the multimedia data according to the short UUID, acquiring the multimedia data from the physical address, and returning the multimedia data to the terminal.

21. A data query method is characterized by comprising an internal network storage system, a network gate and an external network storage system; the intranet storage system comprises a first management node and a plurality of first storage nodes, wherein each first storage node comprises a cloud storage volume and an FTP directory supporting a File Transfer Protocol (FTP) service; the external network storage system comprises a second management node and a plurality of second storage nodes, wherein each second storage node comprises a cloud storage volume and an FTP directory supporting FTP service; an FTP client is operated on the gateway;

22. The method of claim 21 wherein the first management node resolves the first virtual address to obtain the short UUID and a first node identification; redirecting the multimedia data query request to a target first storage node indicated by the first node identification;

23. The method of claim 21 wherein the second management node resolves the second virtual address to obtain the short UUID and a second node identification; redirecting the multimedia data query request to a target second storage node indicated by the second node identification;

and the target second storage node receives the multimedia data query request, acquires a physical address of the multimedia data according to the short UUID, acquires the multimedia data from the physical address, and returns the multimedia data to the terminal.