CN110019233B - Data storage method and system - Google Patents
Data storage method and system Download PDFInfo
- Publication number
- CN110019233B CN110019233B CN201711450257.7A CN201711450257A CN110019233B CN 110019233 B CN110019233 B CN 110019233B CN 201711450257 A CN201711450257 A CN 201711450257A CN 110019233 B CN110019233 B CN 110019233B
- Authority
- CN
- China
- Prior art keywords
- data
- tape library
- management system
- backup server
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1456—Hardware arrangements for backup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2219—Large Object storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data storage method and a data storage system. The method is applied to a data storage system, a backup server in the data storage system is in communication connection with a tape library management system, the tape library management system is in communication connection with a tape library, a near-line storage device and the backup server are in communication connection with data acquisition equipment respectively, and the method stores data to be stored acquired from the data acquisition equipment through the near-line storage device. The backup server obtains the data to be stored from the data acquisition equipment, and backs up the data to be stored in the tape library through the tape library management system. Therefore, the near-line storage and offline storage coexisting storage mode improves the timeliness of recent data reading, and in the process, the backup server does not need to acquire the data to be stored from the near-line storage device in real time, the damage to the near-line storage device is avoided, and the TCO cost of the storage system is effectively reduced.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a data storage method and a data storage system.
Background
The development of communication technology makes people's life gradually enter the information age. The information age brings convenience to the life of people without the need of acquiring, processing and storing data. Among them, storage is one of the most important links, especially for the video surveillance industry. In recent years, as the construction scale of video surveillance increases year by year and the storage time of surveillance data has higher requirements (especially for surveillance data of some special industries), so that the video storage data is exponentially increased, in order to effectively reduce the Total Cost of Ownership (TCO), a tape library is gradually the first choice for storage. However, data stored by using a tape library is convenient for query, so that the user experience is influenced.
Disclosure of Invention
The present invention is directed to a data storage method and system for improving the above-mentioned problems.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a data storage method applied to a data storage system, where the data storage system includes a near-line storage device, a backup server, a tape library management system, and a tape library, the backup server is communicatively connected to the tape library management system, the tape library management system is communicatively connected to the tape library, and the near-line storage device and the backup server are communicatively connected to a data acquisition device respectively, where the method includes: the near line storage device stores data to be stored acquired from the data acquisition equipment; the backup server acquires the data to be stored from the data acquisition equipment; and the backup server backs up the data to be stored in the tape library through the tape library management system.
In a second aspect, an embodiment of the present invention provides a data storage system, where the data storage system includes a near-line storage device, a backup server, a tape library management system, and a tape library, where the backup server is communicatively connected to the tape library management system, the tape library management system is communicatively connected to the tape library, and the near-line storage device and the backup server are communicatively connected to a data acquisition device respectively; the near line storage device is used for storing the data to be stored acquired from the data acquisition equipment; the backup server is used for acquiring the data to be stored from the data acquisition equipment; the backup server is used for backing up the data to be stored in the tape library through the tape library management system.
Further, the backup server is also in communication connection with the near-line storage device, and the tape library management system is further configured to detect data in the tape library according to a preset time period to obtain a detection result; when the detection result is that the data in the tape library is missing, the tape library management system is further configured to obtain the corresponding data from the near-line storage device through the backup server and store the corresponding data in the tape library.
Further, the data storage system further comprises a management platform, wherein the management platform is in communication connection with the near-line storage device; the near-line storage device is further used for feeding back a storage address corresponding to the data to be stored to the management platform after the received data to be stored is stored, so that query and management are facilitated.
Furthermore, the management platform is also in communication connection with the backup server and a tape library management system; the tape library management system is also used for determining corresponding information of data to be recorded according to the detection result; the tape library management system is also used for generating an additional recording request according to the information corresponding to the data to be additionally recorded and sending the additional recording request to the management platform; the management platform is further used for inquiring whether the storage address corresponding to the data to be subjected to additional recording exists according to the additional recording request; when the storage address exists, the management platform is further used for sending the corresponding storage address to the backup server; the backup server is further configured to obtain the corresponding data to be recorded from the near-line storage device according to the corresponding storage address; the backup server is further used for backing up the data to be recorded in the tape library through the tape library management system.
Compared with the prior art, the embodiment of the invention provides a data storage method, and the data to be stored is acquired from the data acquisition equipment through the near-line storage device and the backup server at the same time. The near-line storage device stores the data to be stored, and the backup server backs up the data to be stored in the tape library through the tape library management system. The data to be stored is stored in the near-line storage device to be used as hot data to facilitate query and extraction, and the data to be stored is stored in the tape library to be used as cold data to facilitate storage. The near-line storage and the off-line storage coexist, and the timeliness of recent data reading is improved. In the process, the backup server does not need to acquire the data to be stored from the near-line storage device in real time, so that the damage to the near-line storage device is avoided, and the TCO cost of the storage system is effectively reduced.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of a data storage system according to an embodiment of the present invention.
Fig. 2 is another partial schematic diagram of a data storage system according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a management platform according to an embodiment of the present invention.
Fig. 4 is a flowchart illustrating steps of a data storage method according to an embodiment of the present invention.
Fig. 5 is a timing diagram of a method for upgrading a chain of terminals according to an embodiment of the present invention.
Fig. 6 is another part of a flowchart of steps of a data storage method according to an embodiment of the present invention.
Icon: 100-near line memory devices; 200-a backup server; 300-a tape library management system; 310-a metadata server; 320-a buffer memory array; 400-a management platform; 500-tape library; 600-a data acquisition device; 700-a data storage system; 111-a memory; 112-a processor; 113-communication unit.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
In the related art, in order to solve the problems of long storage time, high query efficiency and the like, a storage system is proposed, in which collected data to be stored is sent to a near-line storage for storage, and the data stored in the near-line storage is transferred to a tape library 500 for storage for a longer time after being changed from hot data to cold data. The scheme has the problems that the performance of a system is reduced, the damage rate of a magnetic disk is improved, the stability of the system is reduced and the like due to the fact that the data reading and the data writing of the magnetic disk array stored in a near line are synchronously carried out. Also, once a tape library 500 is damaged, the missing data is difficult to retrieve. Therefore, the present invention provides a data storage method and system to solve the above problems.
Referring to fig. 1 and fig. 2, a data storage system 700 according to an embodiment of the invention is provided. The data storage system 700 includes a near-line storage device 100, a backup server 200, a tape library management system 300, a management platform 400, and a tape library 500. The backup server 200 is communicatively connected to the tape library management system 300 and the near-line storage device 100, respectively. The tape library management system 300 is communicatively coupled to a tape library 500. The management platform 400 is communicatively connected to the near-line storage apparatus 100, the backup server 200, and the tape library management system 300, respectively. The near-line storage device 100 and the backup server 200 are respectively in communication connection with a data acquisition device 600.
The backup server 200 is configured to convert the received data to be stored into data meeting a preset condition. For example, a video stream is converted to a video file.
The tape library management system 300 includes a meta-server 310 and a buffer storage array 320. The cache memory array 320 is mounted on the meta server 310. The meta server 310 is electrically connected to the backup server 200 and the tape library 500, respectively. The software modules corresponding to the tape library management system 300 are stored in the meta server 310. The meta server 310 may perform writing, reading, checking, etc. of the tape library 500 by running the associated software.
The data acquisition device 600 may be, but is not limited to, a camera, a skynet, a sensory data acquisition device 600, and the like. For convenience of description, in the embodiment of the present invention, the data acquisition device 600 is a camera, and the data to be stored is video stream data.
The tape library 500 is composed of a tape drive and an automatic tape changer in a single machine, and it is possible to pick up a tape from a cassette containing a plurality of cartridges and put it in the drive, or to perform the reverse process. The tape library 500 system is capable of supporting a routine backup process that automatically loads daily backup data onto new tapes.
Fig. 3 is a block diagram of the management platform 400. The management platform 400 includes a memory 111, a processor 112, and a communication unit 113.
The memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The software implementing the data storage method includes at least one software functional module which can be stored in the memory 111 in the form of software or Firmware (Firmware) or solidified in an Operating System (OS) of the management platform 400. The processor 112 is used to execute the executable modules stored in the memory 111.
The Memory 111 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 111 is used to store programs or data. The communication unit 113 is configured to establish a communication connection between the management platform 400 and another communication terminal through the network, and to transceive data through the network.
It should be understood that the architecture shown in fig. 3 is merely a schematic diagram of the architecture of the management platform 400, and that the management platform 400 may also include more or fewer components than shown in fig. 3, or have a different configuration than shown in fig. 3. The components shown in fig. 3 may be implemented in hardware, software, or a combination thereof.
Referring to fig. 4, fig. 4 is a flowchart of a data storage method according to a preferred embodiment of the invention. The data storage method is applied to the data storage system 700. The data storage method comprises the following steps:
in step S101, the near-line storage device 100 stores the data to be stored acquired from the data acquisition apparatus 600.
The near-line storage 100 may be an IPSAN. The aforementioned IPSAN may be a storage network constructed under an IP network. The data is stored in the near-line storage device 100 for convenient query and extraction. The data stored in the near-line memory device 100 is thermal data. It should be noted that the hot data may refer to data that may be frequently accessed by the user in the current time period. In the embodiment of the present invention, the near-line storage device 100 acquires data to be stored from the data acquisition apparatus 600 in real time and stores the data. For example, video stream data is acquired from a camera in real time.
In step S102, the backup server 200 obtains the data to be stored from the data acquisition device 600.
In the embodiment of the present invention, step S101 and step S102 are in parallel relationship, and may be performed synchronously. The backup server 200 acquires the data to be stored from the data acquisition device 600 in real time, and the data to be stored is the same as the data to be stored acquired by the near-line storage apparatus 100.
In step S103, the backup server 200 backs up the data to be stored in the tape library 500 through the tape library management system 300.
In the embodiment of the present invention, the backup server 200 converts the data to be stored into data meeting a preset condition, so as to facilitate storage. For example, if the data to be stored is video stream data, the backup server 200 converts the data into a video file.
The backup server 200 transmits the data converted to satisfy the preset conditions to the tape library management system 300. Specifically, the storage address provided by the cache storage array 320 is obtained by the meta server 310. The above-described buffer storage array 320 constructs a virtual NAS space. The backup server 200 transmits the data converted to satisfy the preset condition to the acquired storage address of the virtual NAS space through the meta server 310.
The tape library management system 300 acquires and stores the inspection parameters corresponding to the data satisfying the preset conditions, so as to facilitate the subsequent inspection of whether the data is valid. The check parameter may be a hash value corresponding to data satisfying a preset condition.
The tape library management system 300 writes the data satisfying the preset condition to the tape library 500. The writing may be performed in such a way that the metadata server 310 sequentially stores the data meeting the preset condition in the buffer storage array 320. The meta server 310 sequentially fetches data satisfying a preset condition from the buffer storage array 320 and writes the data into the tape library 500 according to the operating state of the tape library 500. The operational state of the tape library 500 may be a frequency at which the tape library 500 may receive data writes. The above sequentially taking out the data satisfying the preset condition from the buffer storage array 320 and writing the data into the tape library 500 may be sequentially writing the data in the buffer storage array 320 into the tape library 500 according to the sequence of data storage in the buffer storage array 320 and the writing frequency corresponding to the tape library 500. Therefore, long-time storage of data to be stored is realized.
To facilitate querying data stored in the near-line memory device 100, in other embodiments, the data storage method may include: after the near-line storage device 100 stores the received data to be stored, a storage address corresponding to the data to be stored is fed back to the management platform 400.
In the embodiment of the present invention, each time a data to be stored is stored in the near-line memory device 100, the corresponding storage address is sent to the management platform 400. The management platform 400 generates a query list according to the received storage address, the data to be stored corresponding to the received storage address, and the acquisition time of the data to be stored. To facilitate data querying or retrieval within near-line memory device 100. Referring to fig. 5, taking the data storage system 700 formed by the cameras, the management platform 400, the IPSAN, the backup server 200, the tape library management system 300, and the tape library 500 as an example, the process of storing data may be:
s1: the IPSAN and the backup server 200 acquire video stream data from the cameras simultaneously.
S2: the IPSAN stores the video stream data and feeds back the storage address to the management platform 400.
S3: the backup server 200 transmits the video stream data converted into the video file to the meta server 310 in the tape library management system 300.
S4: the meta server 310 first stores the video file in the buffer storage array 320.
S5: the meta server 310 sequentially fetches the video files stored in the buffer storage array 320 and writes the video files to the tape library 500.
In the embodiment of the present invention, in order to effectively avoid the problem that data is lost and cannot be recovered due to a problem occurring in the tape library 500, as shown in fig. 6, the data storage method provided in the embodiment of the present invention may further include the following steps:
in step S104, the tape library management system 300 detects the data in the tape library 500 according to a preset time period to obtain a detection result.
In one embodiment, the stored verification parameters may be used by the tape library management system 300 to sequentially verify the corresponding data stored in the tape library 500. As another example, a check driver within the tape library management system 300 may also be utilized to check whether a failure point exists in the tape library 500.
In step S105, when the detection result indicates that there is a data missing in the tape library 500, the tape library management system 300 acquires the corresponding data from the near-line storage device 100 through the backup server 200, and stores the corresponding data in the tape library 500.
In the embodiment of the present invention, the tape library management system 300 determines the information corresponding to the data to be recorded according to the detection result. For example, when a failure point is detected in the tape library 500, the corresponding information of the data to be recorded is determined according to the location of the failure point. The corresponding information may be the acquisition time corresponding to the data to be recorded. It should be noted that, while the tape library management system 300 writes the data that satisfies the preset condition and is stored in the buffer storage array 320 into the tape library 500, the correspondence between the position where the data is written, the verification parameter, and the acquisition time of the data to be stored corresponding to the data is stored.
And generating an additional recording request according to the information corresponding to the data to be added and sending the additional recording request to the management platform 400.
The management platform 400 queries whether the storage address corresponding to the data to be supplemented exists according to the supplement request. Specifically, according to the acquisition time corresponding to the data to be additionally recorded, whether the acquisition time of the data corresponding to a storage address is the same as the acquisition time corresponding to the additionally recorded data is inquired from the inquiry list. When the storage address exists, the management platform 400 sends the corresponding storage address to the backup server 200. The backup server 200 obtains the corresponding data to be recorded from the near-line storage device 100 according to the corresponding storage address. The backup server 200 backs up the data to be recorded in the tape library 500 through the tape library management system 300.
It should be noted that there is no precedence order between step S101 and step S105. The execution process does not interfere with each other. In this embodiment, only when the tape library 500 is damaged and data is lost, the corresponding data is requested from the near-line storage device 100, which avoids performing 1: 1, thereby reducing the system load, reducing the damage rate of the near-line storage device 100 and improving the system stability.
The embodiment of the present invention further provides a data storage system 700 corresponding to the above method, and the detailed scheme in the system can be implemented by referring to the above method, which is specifically as follows:
the near-line storage device 100 is configured to store the data to be stored, which is acquired from the data acquisition apparatus 600.
The backup server 200 is configured to obtain the data to be stored from the data acquisition device 600.
The backup server 200 is configured to backup the data to be stored in the tape library 500 through the tape library management system 300.
The tape library management system 300 is further configured to perform detection on data in the tape library 500 according to a preset time period to obtain a detection result.
When the detection result is that the data in the tape library 500 is missing, the tape library management system 300 is further configured to obtain the corresponding data from the near-line storage device 100 through the backup server 200, and store the corresponding data in the tape library 500.
The near-line storage device 100 is further configured to, after storing the received data to be stored, feed back a storage address corresponding to the data to be stored to the management platform 400, so as to facilitate query and management.
The tape library management system 300 is further configured to determine information corresponding to data to be recorded according to the detection result.
The tape library management system 300 is further configured to generate an additional recording request according to the information corresponding to the data to be additionally recorded, and send the additional recording request to the management platform 400.
The management platform 400 is further configured to query whether the storage address corresponding to the data to be supplemented exists according to the supplement request.
When the storage address exists, the management platform 400 is further configured to send the corresponding storage address to the backup server 200.
The backup server 200 is further configured to obtain the corresponding data to be recorded from the near-line storage device 100 according to the corresponding storage address.
The backup server 200 is further configured to backup the data to be recorded in the tape library 500 through the tape library management system 300.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
In summary, the present invention provides a data storage method and system. The data storage method is applied to the system. The method comprises the step of simultaneously acquiring data to be stored from data acquisition equipment through the near-line storage device and a backup server. The near-line storage device stores the data to be stored, and the backup server backs up the data to be stored in the tape library through the tape library management system. The data to be stored is stored in the near-line storage device to be used as hot data to facilitate query and extraction, and the data to be stored is stored in the tape library to be used as cold data to facilitate storage. In the process, the backup server does not need to acquire the data to be stored from the near-line storage device all the time, so that the damage to the near-line storage device is avoided, and the TCO cost of the storage system is effectively reduced.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A data storage method applied to a data storage system, wherein the data storage system comprises a near-line storage device, a backup server, a tape library management system and a tape library, the backup server is communicatively connected to the tape library management system, the tape library management system is communicatively connected to the tape library, the near-line storage device and the backup server are respectively communicatively connected to a data acquisition device, and the backup server is further communicatively connected to the near-line storage device, the method comprising:
the near line storage device stores data to be stored acquired from the data acquisition equipment;
the backup server acquires the data to be stored from the data acquisition equipment;
the backup server backs up the data to be stored in the tape library through the tape library management system;
the tape library management system detects the data in the tape library according to a preset time period to obtain a detection result;
and when the detection result is that the data in the tape library is missing, the tape library management system acquires the corresponding data from the near-line storage device through the backup server and stores the corresponding data in the tape library.
2. The method of claim 1,
the step of the backup server backing up the data to be stored in the tape library through the tape library management system comprises the following steps:
the backup server converts the data to be stored into data meeting preset conditions and sends the data to the tape library management system;
the tape library management system acquires and stores inspection parameters corresponding to the data meeting the preset conditions;
the tape library management system writes the data meeting the preset conditions into the tape library;
the step of the tape library management system detecting the data in the tape library according to a preset time period comprises the following steps: and the tape library management system sequentially verifies the corresponding data stored in the tape library by using the stored verification parameters.
3. The method of claim 2, wherein the tape library management system comprises a meta-server and a buffer storage array, and the step of the tape library management system writing the data satisfying the predetermined condition to the tape library comprises:
the element server sequentially stores the data meeting the preset conditions sent by the backup server in the buffer storage array;
and the element number server sequentially takes out and writes the data meeting the preset conditions into the tape library according to the working state of the tape library.
4. The method of claim 1, wherein the data storage system further comprises a management platform communicatively coupled to the near-line storage device, the method further comprising:
after the received data to be stored is stored by the near-line storage device, the storage address corresponding to the data to be stored is fed back to the management platform, so that query and management are facilitated.
5. The method of claim 4, wherein the management platform is further communicatively coupled to the backup server and a tape library management system, the step of the tape library management system retrieving corresponding data from the near-line storage device via the backup server comprising:
the tape library management system determines corresponding information of data to be recorded according to the detection result;
generating an additional recording request according to the information corresponding to the data to be additionally recorded and sending the additional recording request to the management platform;
the management platform inquires whether the storage address corresponding to the data to be subjected to additional recording exists or not according to the additional recording request;
when the storage address exists, the management platform sends the corresponding storage address to the backup server;
the backup server acquires the corresponding data to be recorded from the near-line storage device according to the corresponding storage address;
and the backup server backs up the data to be recorded in the tape library through the tape library management system.
6. A data storage system is characterized by comprising a near-line storage device, a backup server, a tape library management system and a tape library, wherein the backup server is in communication connection with the tape library management system;
the near line storage device is used for storing the data to be stored acquired from the data acquisition equipment;
the backup server is used for acquiring the data to be stored from the data acquisition equipment;
the backup server is used for backing up the data to be stored in the tape library through the tape library management system;
the magnetic tape library management system is also used for detecting the data in the magnetic tape library according to a preset time period so as to obtain a detection result;
and the tape library management system is further configured to, when the detection result indicates that data in the tape library is missing, acquire, by the backup server, corresponding data from the near-line storage device and store the corresponding data in the tape library.
7. The system of claim 6, wherein the data storage system further comprises a management platform communicatively coupled to the near-line storage device;
the near-line storage device is further used for feeding back a storage address corresponding to the data to be stored to the management platform after the received data to be stored is stored, so that query and management are facilitated.
8. The system of claim 7, wherein the management platform is further communicatively coupled to the backup server and a tape library management system;
the tape library management system is also used for determining corresponding information of data to be recorded according to the detection result;
the tape library management system is also used for generating an additional recording request according to the information corresponding to the data to be additionally recorded and sending the additional recording request to the management platform;
the management platform is further used for inquiring whether the storage address corresponding to the data to be subjected to additional recording exists according to the additional recording request;
the management platform is further configured to send the corresponding storage address to the backup server when the corresponding storage address exists;
the backup server is further configured to obtain the corresponding data to be recorded from the near-line storage device according to the corresponding storage address;
the backup server is further used for backing up the data to be recorded in the tape library through the tape library management system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711450257.7A CN110019233B (en) | 2017-12-27 | 2017-12-27 | Data storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711450257.7A CN110019233B (en) | 2017-12-27 | 2017-12-27 | Data storage method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110019233A CN110019233A (en) | 2019-07-16 |
| CN110019233B true CN110019233B (en) | 2021-07-20 |
Family
ID=67187048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711450257.7A Active CN110019233B (en) | 2017-12-27 | 2017-12-27 | Data storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019233B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112764971A (en) * | 2021-01-25 | 2021-05-07 | 中水三立数据技术股份有限公司 | Data storage backup system and backup method based on fiber channel FC and tape library |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694635A (en) * | 2009-10-22 | 2010-04-14 | 中兴通讯股份有限公司 | Control method and system based on virtual tape library backup and near-line storage node |
| CN102298547A (en) * | 2011-09-20 | 2011-12-28 | 杭州华三通信技术有限公司 | Data backup method and device |
| CN103092952A (en) * | 2013-01-15 | 2013-05-08 | 深圳市连用科技有限公司 | Storage system and management method of massive unstructured data |
| CN103365972A (en) * | 2013-06-27 | 2013-10-23 | 北京中科金财科技股份有限公司 | Intelligent processing system for mass data |
| CN107015884A (en) * | 2016-01-28 | 2017-08-04 | 杭州海康威视数字技术股份有限公司 | A kind of date storage method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5434705B2 (en) * | 2010-03-12 | 2014-03-05 | 富士通株式会社 | Storage device, storage device control program, and storage device control method |
| CN106484330A (en) * | 2016-09-27 | 2017-03-08 | 郑州云海信息技术有限公司 | A kind of hybrid magnetic disc individual-layer data optimization method and device |
| CN206741467U (en) * | 2017-04-10 | 2017-12-12 | 四川慧存大数据存储科技有限公司 | Intelligent digital profile storage system |
-
2017
- 2017-12-27 CN CN201711450257.7A patent/CN110019233B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694635A (en) * | 2009-10-22 | 2010-04-14 | 中兴通讯股份有限公司 | Control method and system based on virtual tape library backup and near-line storage node |
| CN102298547A (en) * | 2011-09-20 | 2011-12-28 | 杭州华三通信技术有限公司 | Data backup method and device |
| CN103092952A (en) * | 2013-01-15 | 2013-05-08 | 深圳市连用科技有限公司 | Storage system and management method of massive unstructured data |
| CN103365972A (en) * | 2013-06-27 | 2013-10-23 | 北京中科金财科技股份有限公司 | Intelligent processing system for mass data |
| CN107015884A (en) * | 2016-01-28 | 2017-08-04 | 杭州海康威视数字技术股份有限公司 | A kind of date storage method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110019233A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220327026A1 (en) | Partial database restoration | |
| US8418003B2 (en) | Log collecting apparatus, program, method, and system | |
| CN103167026B (en) | A kind of cloud store environmental data processing method, system and equipment | |
| JP6521462B2 (en) | Improving backup system performance | |
| CN104281533B (en) | A kind of method and device of data storage | |
| US7987153B2 (en) | Apparatus and method for automatically migrating user's working data | |
| CN108093207B (en) | Data storage and acquisition method and device | |
| US20140244583A1 (en) | Method of Processing File changes and Electronic Device Thereof | |
| US6397247B1 (en) | Failure prediction system and method for a client-server network | |
| CN105376277A (en) | Data synchronization method and device | |
| CN109522316A (en) | Log processing method, device, equipment and storage medium | |
| CN103678143A (en) | File storage method and device and electronic equipment | |
| CN103986779A (en) | Application data transmission method, device and terminal equipment | |
| CN105550230B (en) | The method for detecting and device of distributed memory system node failure | |
| US9916220B2 (en) | Smart logging of trace data for storage systems | |
| CN107133334B (en) | Data synchronization method based on high-bandwidth storage system | |
| EP3822763B1 (en) | Data reading method, device, system, and distributed system | |
| CN110019233B (en) | Data storage method and system | |
| CN106487852B (en) | Method, device, terminal equipment and system for realizing client file synchronization | |
| CN112925672B (en) | Data recovery method, device, equipment and storage medium | |
| CN113468274A (en) | Data storage method and device, storage medium and electronic equipment | |
| CN110515803B (en) | Processing method and device for log message and electronic equipment | |
| CN114238018B (en) | Method, system and device for detecting integrity of log collection file and storage medium | |
| CN114281769A (en) | Method and device for managing files on disk, computer equipment and storage medium | |
| CN115543939A (en) | Log data storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |