CN116089427A - Management method and system for multi-medium fusion storage of electronic files - Google Patents

Abstract

The application discloses a management method and a system for multi-medium fusion storage of electronic files, and belongs to the technical field of file management. The method comprises the following steps: acquiring a management request input by a user; corresponding management operations are executed on three storage devices of distributed storage, an optical disc library and a tape library and electronic files stored on the storage devices according to management requests, wherein the distributed storage uses a magnetic disc as a storage medium, the optical disc library uses an optical disc as a storage medium, the tape library uses a tape as a storage medium, and the management operations comprise visual management of storage data, visual management of the storage devices, storage data inspection and storage log management; and displaying the result of the management operation on the front-end interface. By adopting the management method, the electronic archive data of various storage devices and electronic archive data stored on the storage devices for a long time are managed in a centralized and unified mode, and meanwhile, transparent management is realized through the management visualization function, so that the management quality of the electronic archive is effectively improved.

Description

Management method and system for multi-medium fusion storage of electronic files

Technical Field

The application belongs to the technical field of file management, and particularly relates to a management method and system for multi-medium fusion storage of electronic files.

Background

Along with the rapid development of social informatization, the process of digitizing the paper file is also continuously accelerated, so that the quantity of the electronic file data is rapidly increased, and the construction of digital file resources is promoted.

At present, most of nationwide archive digital resource construction standards are not uniform, most archive departments adopt different storage media to realize the storage of electronic archives, so that data storage formats are various, a backup mechanism is not sound, a plurality of dead archives are formed, potential safety hazards exist, system management of the electronic archives and sharing utilization of archive digital resources are not facilitated, the development and utilization level of archive information resources is low, and archive values are not fully exerted.

Therefore, how to improve the management quality of electronic files stored on various media is an important issue to be solved.

Disclosure of Invention

The embodiment of the application provides a management method and a system for multi-medium fusion storage of electronic files, which can uniformly manage electronic file data stored in various storage media and improve the management quality of the electronic files.

In a first aspect, an embodiment of the present application provides a method for managing multimedia fusion storage of electronic files, where the method includes: acquiring a management request input by a user; executing corresponding management operation on three storage devices of distributed storage, an optical disc library and a tape library and electronic files stored on each storage device according to a management request, wherein the distributed storage uses a magnetic disc as a storage medium, the optical disc library uses the optical disc as the storage medium, and the tape library uses a tape as the storage medium; the management operation comprises visual management of storage data, visual management of storage equipment, storage data checking and storage log management; and displaying the result of the management operation on the front-end interface.

In a possible implementation manner of the first aspect, the storing data visual management includes querying a total data amount of the electronic file and data amounts of various electronic files, searching the electronic file according to any data information of the electronic file, obtaining storage equipment and other data information of the electronic file, and operating the electronic file; the visual management of the storage devices comprises checking the storage condition of each storage device and monitoring the operation state of each storage device, wherein the storage condition of the storage device comprises total capacity, used capacity and storage duty ratio, and the operation state of the storage device comprises the health state and the utilization rate of a central processing unit (Central Processing Unit, CPU), a memory or a network card; the stored data checking comprises the steps of detecting different frequencies of electronic archive data of different importance levels stored on different storage devices according to a preset stored data checking strategy; storage log management includes at least one of viewing, downloading, or exporting storage logs.

In a possible implementation manner of the first aspect, monitoring an operation state of each storage device includes: monitoring the health state of a CPU, a memory or a network card of each storage device, and displaying an alarm prompt on a front-end interface when the CPU, the memory or the network card fails; setting the utilization rate threshold of the CPU, the memory or the network card of each storage device according to the characteristics of each storage device, and displaying an alarm prompt on the front end interface when the utilization rate of the CPU, the memory or the network card is monitored to exceed the threshold.

In a possible implementation manner of the first aspect, when performing the visual management of the stored data, the method includes the following steps: and inquiring the electronic files stored on each storage device through the metadata information reported by the distributed storage, the optical disc library and the tape library.

In a possible implementation manner of the first aspect, when performing the visual management of the storage device, the method includes the following steps: checking capacity information of each storage device through a data capacity inquiry interface provided by distributed storage, an optical disc library and a tape library; the running state of each storage device is checked and monitored through the health state query interface and the real-time utilization rate query interface of the CPU, the health state query interface and the real-time utilization rate query interface of the memory or the health state query interface and the real-time utilization rate query interface of the network card, which are provided by the distributed storage, the optical disc library and the magnetic tape library, so that the visual management of the storage device is realized.

In a possible implementation manner of the first aspect, when performing the stored data check, the method includes the steps of: and generating a data checking task according to the stored data checking strategy, receiving the data checking task by the distributed storage, the optical disc library and the tape library and automatically executing the data checking task, thereby realizing the stored data detection.

In a possible implementation manner of the first aspect, when performing storage log management, the method includes the following steps: and searching, checking, downloading or exporting the user operation log and the system operation log of each storage device through the log access interfaces provided by the distributed storage, the optical disc library and the tape library, thereby realizing the storage log management.

In a possible implementation manner of the first aspect, the management operation further includes storage device function management, where the storage device function management is to perform unified management on storage functions of each of three types of storage devices, and is classified into distributed storage function management, optical disc library function management, and tape library function management according to differences of the storage devices.

In a second aspect, an embodiment of the present application provides a management system for multi-media fusion storage of electronic files, where the system includes: the acquisition module is used for acquiring a management request input by a user; the service management module is used for executing corresponding management operation on three storage devices of the distributed storage, the optical disc library and the tape library and the electronic files stored on each storage device according to the management request; the distributed storage uses a magnetic disk as a storage medium, an optical disk library uses an optical disk as a storage medium, and a magnetic tape library uses a magnetic tape as a storage medium; the management operation comprises visual management of storage data, visual management of storage equipment, storage data checking and storage log management; the display module is used for displaying the result of executing the management operation; the storage module is used for storing the electronic files and comprises three storage devices, namely distributed storage, an optical disc library and a tape library.

In a possible implementation manner of the second aspect, the service management module includes: the storage data visual management module is used for inquiring the total data volume of the electronic files and the data volume of various electronic files, searching the electronic files according to any data information of the electronic files, obtaining storage equipment and other data information of the electronic files, and operating the electronic files; the visual management module of the storage equipment is used for checking the storage condition of each storage equipment and monitoring the operation state of each storage equipment, wherein the storage condition of the storage equipment comprises total capacity, used capacity and storage duty ratio, and the operation state of the storage equipment comprises the health state and the utilization rate of a CPU (central processing unit), a memory or a network card; the storage data checking module is used for detecting different frequencies of the electronic archive data of different importance levels stored on different storage devices according to a preset storage data checking strategy; and the storage log management module is used for at least one of checking, downloading or exporting the storage log.

Compared with the prior art, the embodiment of the application has the beneficial effects that: according to the management method for the multi-medium fusion storage of the electronic files, the management request is firstly obtained, then the management operation corresponding to the request is executed by accessing the three storage devices and the electronic files stored on the three storage devices, the operation result is displayed on the front-end page, the various storage media and the data stored on the storage media are safely stored and centrally and uniformly managed by the method, meanwhile, the management visualization enables the whole management process to be clearer and more transparent, the management level and the development and utilization level of the electronic files are effectively improved, the sharing and the utilization of file digital resources are facilitated, and the management quality of the electronic files is effectively improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an application scenario diagram of an electronic archive multimedia fusion storage management method provided in the present application.

FIG. 2 is a flow chart of a method for managing multi-media fusion storage of electronic files according to the present application.

Fig. 3 is a schematic diagram illustrating a classification of management operations of another electronic archive multimedia fusion storage management method provided in the present application.

Fig. 4 is a schematic diagram of visual management of stored data in a management method provided in the present application.

Fig. 5 is a schematic diagram of visual management of stored data in another management method provided in the present application.

Fig. 6 is a schematic diagram of visual management of a storage device in a management method provided in the present application.

Fig. 7 is a schematic diagram of visual management of a storage device in another management method provided in the present application.

Fig. 8 is a schematic diagram of storage log management in a management method provided in the present application.

Fig. 9 is a logic diagram of a data slice in a management method according to the present application.

Fig. 10 is a schematic diagram of a read-only mode in a management method provided in the present application.

Fig. 11 is a schematic diagram of trusted identification management in a management method provided in the present application.

Fig. 12 is an expanded schematic diagram of a distributed storage node in a management method provided in the present application.

Fig. 13 is a schematic diagram of a storage server cluster in a management method provided in the present application.

Fig. 14 is a schematic view of visual management of stored data in a management method provided in the present application.

FIG. 15 is a schematic diagram of an electronic archive multi-media fusion storage management system according to the present application.

FIG. 16 is a schematic diagram illustrating a structure of another electronic archive multi-media fusion storage management system provided in the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The embodiment of the application relates to a management method of electronic files suitable for fusion storage of multiple storage media, and fig. 1 is a schematic diagram of an application scenario of the method, as shown in fig. 1, and the management method is applied to a unified management system based on a three-in-one fusion storage technology. The unified management system comprises a client layer, a service layer and a data layer, wherein the client layer comprises a plurality of clients positioned at the front end and is used for receiving a request input by a user and displaying an output result of management; the service layer comprises a management server which is responsible for realizing the main functions of management service, when the front end needs to perform data access and other management operations, the management server calls the data in the storage database and performs operations such as sorting analysis and the like, and the processed information is fed back to the front end; the storage database contains stored electronic archive data which are stored in three storage devices of a distributed storage device, an optical disc library and a tape library according to factors such as types or characteristics, and are backed up on different storage media, so that the safety of the electronic archive is effectively improved.

The management method of the multi-medium fusion storage of the electronic files can realize centralized and unified management of the electronic file data and three storage devices on three storage media, and is favorable for safe storage, convenient utilization and resource sharing of the files; meanwhile, the management method can achieve the visual effect, so that management staff can see the managed data and equipment at a glance, the problems that the original optical disk and tape are used for carrying out data backup and then become a black box, whether the backup content and the medium have redundancy, damage and the like are not known are solved, and the quality of file management is improved.

Fig. 2 shows a management method for multi-media fusion storage of electronic files provided by the application, which includes the following steps.

S1, acquiring a management request input by a user.

Step S1 may be performed by the acquisition module 1 of the management system in fig. 15.

S2, executing corresponding management operation on the three storage devices, namely the distributed storage device, the optical disc library and the tape library, and the electronic files stored on the storage devices according to the management request.

The distributed storage is a virtual storage device formed by storing data in a distributed manner on a plurality of disks of storage servers adopting disk array technology and connecting the servers through a network. An optical disc library is a storage device that uses optical discs as a primary storage medium. Tape libraries are storage devices that use tape as the primary storage medium.

After the target management request is obtained, the required data of each storage device or the required data of the electronic file in the storage device are called through various interfaces which are arranged, and then the corresponding processing of the target request is carried out on the called data, such as data arrangement, data modification and the like, so as to achieve the management purpose.

Step S2 may be performed by the service management module 2 and the storage module 4 of the management system in fig. 15.

And S3, displaying the result of the management operation on the front-end interface.

After corresponding management operation is executed according to the acquired management request, the result after the management operation is executed is fed back to the front-end display interface, and a user can acquire required information or execute subsequent operation according to the result displayed on the interface.

Step S3 may be performed by the display module 3 of the management system in fig. 15.

Fig. 3 illustrates the type of the management operation in step S2 of the present application, as shown in fig. 3, the object of the management operation performed according to the management request may be a storage device, storage data, or the whole management system, and the specific content of the management operation may include storage data visual management, storage device visual management, storage data inspection, and storage log management.

In one embodiment, the visual management of the stored data may include querying data volume information of the electronic files, that is, querying total data volume of the electronic files and data volumes of various types of electronic files stored in all storage devices, querying total data volume of the electronic files and data volumes of various types of electronic files in each storage device, and analyzing growth rate conditions of the electronic file data according to the queried data volume information of the electronic files.

When the obtained management request is to inquire the data volume information of the electronic file, the management operation is performed, and the inquiry result or the analysis result is displayed on the front end interface, wherein the display interface can be seen in fig. 4. Through the process, the data quantity and the data growth condition can be checked and analyzed, so that a user can conveniently and rapidly collect and schedule the data, and the stored electronic archive data can be efficiently utilized.

In addition, the visual management of the stored data can also include inquiring and managing a specific electronic file, namely searching a corresponding electronic file according to any data information of the electronic file, further obtaining storage equipment and other data information of the electronic file, and further carrying out management of referring to the electronic file, editing the data information and the like on the basis.

When the obtained management request is to query and manage a specific electronic file, the management operation is performed, and the query result or management mode is displayed on the front-end interface, where the display interface can be see fig. 5. The electronic files are classified according to the data information thereof, and the stored electronic file data can be conveniently and quickly found and utilized when the above-mentioned process is executed.

The data information of the electronic file includes, but is not limited to, electronic file sequence number, file number, responsible person, title, date, security level, name or remark.

In one embodiment, when the visual management of the storage data is performed, the steps include querying the electronic files stored on each storage device through metadata information reported by the distributed storage, the optical disc library and the tape library.

In one implementation, the distributed storage, the optical disc library and the tape library all report the stored metadata information in an extensible markup language (EXtensible Markup Language, XML) format, wherein the metadata information comprises data information of an electronic file, namely, information such as an electronic file sequence number, a file number, a responsible person, a title, a date, a secret level, a name or a remark. Optionally, the format of the metadata information on the XML store is pending.

In one embodiment, the visual management of storage devices may include viewing the storage of each storage device and monitoring the operational status of each storage device.

The storage conditions of the storage device may include total capacity, used capacity, and storage duty cycle. The running state of the storage device comprises the health state and the utilization rate of the CPU, the memory or the network card.

In one implementation manner, monitoring the operation state of each storage device in the visual management of the storage device may include monitoring the health states of the CPU, the memory or the network card of the three storage devices including the distributed storage, the optical disc library and the tape library, and when the CPU, the memory or the network card fails, displaying an alarm prompt on the front-end interface, for example, "the CPU of the distributed storage fails". The management operation realizes the health status early warning of hardware such as a CPU, a memory or a network card of a disk storage, an optical disk storage and a tape storage hardware server, and when equipment fails, the alarm prompt can be timely carried out through a platform interface, so that users can be ensured to quickly find out the wrong server and hardware, and the availability of the system is improved.

In addition, monitoring the running state of each storage device may also include setting a threshold of the utilization rate of the CPU, the memory or the network card of each storage device according to the characteristics of the three storage devices, and displaying an alarm prompt, for example, an "memory overload of an optical disc library" on the front-end interface when the utilization rate of the CPU, the memory or the network card is monitored to exceed the preset threshold. According to the management operation, relevant technical parameter indexes are set according to the characteristics of different storage devices, when the hardware resources exceed preset values in the use process, the system can rapidly alarm and feed back to the platform, and rescue and repair can be timely carried out before the hardware resources are in a problem.

When the acquired management request is for checking or managing the storage condition and the running state of a certain storage device, the management operation is performed, and the query result or the management mode is displayed on the front-end interface. Fig. 6 shows a page for displaying a set of storage conditions and running states of three storage devices, and the overall conditions of the three storage devices can be seen through fig. 6. Specifically, the specific situation of each storage device can be referred to by clicking, as shown in fig. 7, in order to view and manage one example of a display interface of the storage situation and the running state of the distributed storage.

In one embodiment, when the abnormal running state of the storage device is monitored, the user can be notified through a built-in mail system, so that the user can be ensured to acquire abnormal information in time and perform relevant treatment.

In one embodiment, when the visual management of the storage device is executed, the method includes the steps of checking capacity information of each storage device through a data capacity inquiry interface provided by a distributed storage, an optical disc library and a tape library, and checking and monitoring an operation state of each storage device through a health state inquiry interface and a real-time utilization inquiry interface of a CPU, a health state inquiry interface and a real-time utilization inquiry interface of a memory or a health state inquiry interface and a real-time utilization inquiry interface of a network card provided by the distributed storage, the optical disc library and the tape library, so that the visual management of the storage device is realized.

In one embodiment, to implement visual management of the distributed storage, each storage node of the distributed storage provides a health status and utilization query interface for the CPU, memory, and network card. The utilization rate and the health state of the CPU, the memory and the network card of the storage node can be monitored through the node query interface.

The distributed storage also provides a real-time monitoring storage node interface, and the monitoring and recording of whether the disk is on line or not and the disk capacity can be realized through the real-time monitoring storage node interface, so that the status information such as the fault node and the monitoring disk can be timely obtained, the fault point can be conveniently and quickly found and positioned by a user, the operation such as maintenance and equipment replacement can be timely carried out on the abnormal node, and the data safety can be guaranteed to the greatest extent.

The distributed storage also provides a system node list query interface, and the detection and monitoring of the real-time monitoring storage nodes can be realized through the system node list query interface.

The distributed storage also provides a storage pool information query interface, and the storage pool information query interface can be used for querying whether the storage pool is online, the storage pool capacity and other information, monitoring the state of the storage pool in real time, ensuring that the storage pool can display faults at the first time after offline, facilitating users to discover and locate fault points in time, relieving offline faults in time and ensuring data safety. Specifically, the distributed storage also provides a data capacity query interface, and specifically, the total space, the used space and the residual space of the storage pool can be checked through the interface, so that the front-end interface can acquire information in real time, and a client can more intuitively see the use state of the storage pool.

In one embodiment, in order to realize visual management of the optical disc library, the optical disc library provides an optical disc library health state query interface, specifically, the monitoring condition of the optical disc library can be tracked and invoked through the optical disc library health state query interface, abnormal information of the optical disc library can be found in time, and high availability of optical disc library hardware is ensured.

The optical disc library also provides a CPU real-time utilization rate query interface, the CPU utilization rate in the optical disc library system can be checked and called through the CPU real-time utilization rate query interface, CPU abnormal information of the optical disc library can be found timely, and a user can find hardware abnormal equipment information timely.

The optical disc library also provides a memory real-time utilization rate query interface, can realize summarized query and call of the memory service condition through the memory real-time utilization rate query interface, timely discovers abnormal information, and performs relevant treatment.

The optical disc library also provides a network bandwidth real-time utilization rate query interface, can track and query network bandwidth information through the network bandwidth real-time utilization rate query interface, and can perform anomaly analysis according to anomaly data stored in the system when network anomalies occur, so as to discover error information in time.

The optical disc library also provides an optical disc library information inquiry interface, and the optical disc library capacity, the number of optical drives, the optical drive state, the number of optical disc slots, the number of optical discs, the optical disc state, the optical disc type, the optical disc capacity and other data can be inquired and utilized through the optical disc library inquiry interface, so that a user can quickly know the information of a driver, an optical disc, slots and the like in the optical disc library, and quickly know the service condition of the optical disc library. Specifically, the optical disc library also provides a data capacity query interface, and the interface can be used for checking the total space, the used space, the residual space and other data information of the storage pool, so that the front-end interface can acquire the stored data information, the stored data capacity and the like in real time, a user can quickly and intuitively see the use state of the storage pool, and the operations such as data expansion and the like can be conveniently and timely performed.

In one embodiment, to realize visual management of the tape library, the tape library provides a tape library health status query interface, in particular, tracking and recording of the monitoring condition of the tape library can be performed through the tape library health status query interface, so that abnormal information of the tape library can be found in time, and high availability during use of the tape library is ensured.

The tape library also provides a CPU real-time utilization rate query interface, the use condition of the CPU can be timely obtained through the CPU real-time utilization rate query interface, and when the CPU works abnormally, the technology is used for carrying out relevant treatment, so that the high availability of the system is improved.

The tape library also provides a memory real-time utilization rate query interface, and can timely acquire the service condition information of the memory through the memory real-time utilization rate query interface, so that equipment and tasks with high memory utilization rate can be quickly found, and the availability of the library system is improved.

The tape library also provides a network bandwidth real-time utilization rate query interface, and can realize the rapid tracking of the utilization rate of the internal network through the network bandwidth real-time utilization rate query interface, locate abnormal flow data in time and perform relevant treatment.

The tape library also provides a tape library information inquiry interface, and the tape library information inquiry interface can rapidly realize the tracking and inquiry of the storage data of the tape library such as whether the storage medium is on line, the total capacity of the tape library, the number of drivers, the types of the drivers, whether the drivers are on line, the number of available slots, the number of tape cassettes, the total occupied data volume and the like, and is convenient for users to use. In particular, the tape library also provides a data capacity query interface, which may be an application programming interface (Application Programming Interface, API) that supports the information content of calls including: tape library slot information, tape model information, available empty tape information, tape available capacity information, tape data occupancy information, and the like. The information is quickly called through the API interface, so that the required information, the statistical summary data capacity information and the total available capacity information can be quickly obtained, and the quick docking and calling are convenient.

In one embodiment, the stored data inspection may include detecting the electronic archive data of different importance levels stored on different storage devices with different frequencies according to a preset stored data inspection policy.

In one implementation, the media of mainstream brands and specifications on the market can be acquired in advance, and the performance indexes of the media are summarized. When the storage medium is used for storage, the storage position of the file can be optimized according to the parameters, characteristics, life cycle, distributed management enabled or not and other information of the storage medium, and the storage rule of the medium can be adjusted by updating the system parameters in the later period. When data inspection is performed, the detection frequency of the carrier is preset according to the form of the carrier (magnetic disk, optical disk and magnetic tape) and the ring characteristics of each form of carrier (for example, the magnetic disk can be rewritten for many times and the magnetic tape can be written once basically). The frequency of detection of the electronic file is calculated based on different importance levels, such as confidentiality level, cao Cun duration, whether structured data is included, whether multimedia files are included, etc. And forming a preset stored data checking strategy according to the detection frequency of the carrier and the detection frequency of the electronic file, wherein the stored data checking strategy can realize a configuration strategy aiming at multiple media types and multiple file levels, and periodically and regularly detecting the file data in the medium. And carrying out heterogeneous comparison on the electronic archive data with different importance levels according to different strategies, so as to ensure the consistency and tamper resistance of the archive data in the multimedia.

When data checking is carried out, a stored data checking strategy is issued to each storage device, and the strategy setting parameters comprise a time range, a unit range and a gear number range. And searching the corresponding electronic file in the storage device according to the setting parameters. The folder-defining position of the user data includes time, unit and file number information at the time of storage. Stored data inspection supports electronic archive four-way detection functions including authenticity detection, integrity detection, availability detection, security detection, etc., while supporting project detection policy configuration.

In one embodiment, when the stored data inspection is performed, the steps include generating a data inspection task according to a stored data inspection policy, and receiving the data inspection task by the distributed storage, the optical disc library and the tape library and automatically performing the data inspection task, thereby implementing the stored data inspection.

In one embodiment, to enable inspection of data stored in a distributed store, the distributed store receives a data inspection task via an XML file and automatically performs the task. In one implementation, a cryptographic algorithm of a cryptographic function of the cryptographic SM3 cryptographic function that specifies data is automatically generated based on a data inspection task, and automatically comparing with the SM3 hash function cryptographic algorithm of the original national password stored in the data, and if the comparison is consistent, checking is passed, otherwise, generating a data checking error report.

In one embodiment, to enable inspection of data stored in the optical disc library, the optical disc library receives a data inspection task in an XML file manner and automatically performs the task. In one implementation, a cryptographic algorithm of a cryptographic function of the cryptographic SM3 cryptographic function that specifies data is automatically generated based on a data inspection task, and automatically comparing with the original SM3 stored in the data, and if the comparison is consistent, checking is passed, otherwise, generating a data checking error report.

In one embodiment, to enable inspection of data stored in a tape library, the tape library provides an API interface and a web-based, distributed modular component (WebSevice) call to perform a data inspection call task and automatically execute the task, copy data in tape to disk cache, and provide user calls.

In one embodiment, storage log management includes at least one of viewing, downloading, or exporting storage logs.

In one embodiment, in performing the storage log management described above, the steps include: and searching, checking, downloading or exporting the user operation log and the system operation log of each storage device through the log access interfaces provided by the distributed storage, the optical disc library and the tape library, thereby realizing the storage log management.

User operation log management and system operation log management can provide relevant log information for user behavior audit and system fault diagnosis. The user operation log can record management operations corresponding to the management requests, further can analyze the use information of the client, count the access module and the information on the same day, including information reading times, writing times and deleting times, and can determine a responsible person through inquiring the user operation log once the conditions of data loss or secret leakage and the like occur. The system operation log can also provide basis for monitoring the management system. For example, the system operation log may be classified into different grades, and then a system abnormal operation alarm may be provided according to the grades, and in particular, notification may be performed through a short message or mail.

In one embodiment, to implement log management for distributed storage, the distributed storage provides an access interface for user operation logs, system running logs. Through the interface, the functions of searching, checking history and the like can be provided according to the log time, and important logs can be downloaded and exported.

In one embodiment, to implement log management of the optical disc library, the optical disc library also provides an access interface for user operation log and system operation log. The support of the user operation log access interface enables the operation log of the optical disc library management system to be output and downloaded according to the time sequence; the system operation log access interface enables the system log of the optical disc library to be output and downloaded according to the level.

In one embodiment, to enable log management of a tape library, a tape library system is provided that uses a log interface.

When the obtained management request is to view or manage the storage log, the management operation is performed, and the query result or the management mode is displayed on the front-end interface, where the display interface can be see fig. 8.

In one embodiment, automatic expansion of the storage device may be achieved based on monitoring of the storage log. Specifically, expanding a storage device has several mechanisms: 1. expanding the capacity by adding a new storage medium; 2. expanding the capacity through compressed data, and selecting a compressed data target according to a log management mechanism; 3. and deleting the data to expand the capacity by degrading part of the data storage level. Such as non-critical data, has a shelf life approaching that of the expired data, and is preferentially rejected from the redundancy mechanism.

The management operation comprises the visual management of the storage data, the visual management of the storage equipment, the storage data inspection and the storage log management, and also comprises the function management of the storage equipment, namely, the electronic archive multi-medium fusion storage management method provided by the application can realize the management of other functions of the storage equipment besides the management of the storage function of the storage equipment according to the request.

The object of storage function management according to management can be divided into distributed storage function management, optical disc library function management and tape library function management.

The distributed storage function management may include storage security policy management, storage capacity expansion management, storage load balancing management, storage data read-write audit management, storage process trusted identification, and long-term reliability design.

In one embodiment, storage security policy management includes providing different redundancy policy storage mechanisms (e.g., a.gtoreq.2 copy of data mechanism, a data cross-node slice storage mechanism) for different electronic archive data, while providing electronic archive data store-and-read-traceback audit and electronic archive data read-only attribute management.

In one implementation, the distributed storage may provide storage modes such as Erasure Code (EC), copy, etc., and when the disk is dropped or damaged, the front-end archive service usage is not affected. The node level security technology is adopted, so that the data of a node with faults is not lost and the application is not stopped. More than one backup of the electronic archive data can be realized in the storage system, and the data loss can not occur when any storage node is broken.

In one implementation, when the distributed storage uses the erase Code mode to store data, as shown in fig. 9, different data protection modes can be configured for the directory/file, the system is divided into a plurality of original data strips according to the setting proportion, then M redundant data strips are calculated for each N original data strips, and the n+m strips form a stripe and are written into the storage system. When the system fails and some stripes are lost, normal data reading and writing can be performed as long as the number of the lost stripes in one stripe does not exceed M. The missing stripes may be calculated from the remaining stripes by a data recovery algorithm. In this way, the space utilization is about N/(n+m), and the reliability of the data is determined by the magnitude of the M value, and the greater M, the higher reliability.

In one implementation, the distributed storage may provide a read-only mode of turning on electronic archive data, such as FIG. 10, which is read only and cannot be modified and rewritten once the electronic archive data is set to read-only mode.

In one implementation, the distributed storage may provide a log-based auditing technique, where after clients use network file systems (Network File System, NFS), universal network file systems (Common Internet File System, CIFS), protocol mount storage pools, real-time statistics and logging of each client use store is performed, as in fig. 11, as a data traceability basis when data loss occurs or use is stolen by others. Statistical data were retained by default for 30 days. By establishing a storage access channel with a client or standard protocol connection such as NFS, the auditable content is as follows: the method comprises the steps of supporting file creation, directory deletion, data writing, data reading, renaming and real-time recording; support Log files (Log), electronic Mail (E-Mail), touch screen, mobile phone data logging alarms and notifications.

In one embodiment, the storage capacity expansion management includes expanding the distributed storage according to storage conditions of the storage device and log monitoring conditions.

The storage system has elastic capability, meaning that users can flexibly increase resources in the data storage pool according to business needs without interrupting system operation. One of the design targets of the online expansibility is elasticity, which allows dynamic addition of data volumes, expansion of storage servers and the like, without affecting the normal operation and business service of the system. The online expansibility design has quite sufficient elasticity, can meet the application requirement on high elasticity of a storage system, and has greater significance especially for a cloud storage service system. On-line extensibility design achieves this design goal primarily through storage virtualization technology and logical volume management.

Linear expansion is very difficult to achieve for storage systems, and is typically LOG logarithmic between system scale expansion and performance improvement, since some of the performance improvement is consumed by the corresponding load that is generated. Many parallel/cluster/distributed file systems at present have very high expansion capability, the number of storage nodes of the distributed file system (Lustre) can reach more than 1000, the number of clients can reach more than 25000, and the expansion capability is very strong, but Lustre is not linearly expanded.

The vertical expansion (Scale-Up) aims to improve the storage capacity or performance of a single node, and often has various restrictions in theory or physically, but cannot meet the storage requirement. The expansion mechanism is a current storage technology hot spot, and can effectively meet the storage requirements of capacity, performance and the like. Current parallel/clustered/distributed file systems mostly have lateral expansion capability.

The online extensibility design is a linear lateral extension architecture, which can achieve linear storage capacity and performance improvement by laterally extending storage nodes. In one implementation manner, when the capacity expansion of the distributed storage is implemented, the capacity expansion can be achieved by combining the longitudinal expansion on-line expansion design, the storage capacity can be increased by adding the disk of each node, the storage node is increased to improve the performance, as shown in fig. 12, the operation of adding the storage node is implemented through a preset management path, so that more disk, memory and Input/Output (I/O) resources are aggregated into a virtual storage pool with larger capacity and higher performance.

In one embodiment, storage load balancing includes eliminating hot spot data problems that are common in conventional storage systems due to uneven loading by implementing concurrent data access support for multiple storage servers. The storage pool cluster storage realizes an efficient and comprehensive load balancing function through striped distribution of data in the storage server cluster. As shown in fig. 13, the above-mentioned load balancing management may be implemented through a preset management approach.

In one embodiment, the long-term reliability design includes a security mechanism that provides copies of files that may enable security at the node level.

The distributed cloud storage system provides a security guarantee mechanism for file copies, and can realize security protection at the node level. In one implementation, the data security implementation mechanism is as shown in the storage server cluster in FIG. 14: the distributed cloud storage system stores file segments on storage servers, wherein each segment has more than 2 copies stored on different storage servers. 2 copies of the file ' file block 3 ' are respectively stored on the hard disk D1 of the storage server 1 and the hard disk D2 of the storage server 2, when the storage server 2 is down or the hard disk D2 fails, the user can still read the file ' file block 3 ' of the file from the D1 of the storage server 1, and meanwhile, the system automatically generates a new copy of the file block 3 ' on the D2 of the other storage server 3.

The data security mechanism enables the distributed cloud storage system to automatically recover data as long as the capacity of the residual space in the system is larger than the capacity of the data stored in the lost hardware. Since the other copy of the data on each storage device is distributed on all other storage devices (except the other storage devices of the same storage server), the data recovery is also to the intact storage devices in the system, so the recovery process of the storage system is a multi-to-multi recovery process, and the recovery speed is greatly higher than that of the traditional storage system. Automated failure recovery and high speed recovery speed greatly improve the reliability of the storage system.

Optical disc library function management may include optical storage read-write management and system operation and maintenance management.

In one embodiment, optical storage read-write management includes supporting extra-long directory depth and file name length, with a maximum file name length of 255 bytes and a maximum directory depth exceeding 3900 bytes, supporting Chinese. The optical disc file format supports universal devices, and the standard blue-ray optical disc drive of a third party can read normally. The recorded optical disc can read data through a standard blue-ray optical drive of a third party, and when the optical disc library is not available, a user can conveniently check the data. Support multiple CD-ROM drives and write and read concurrently. The optical disc library supports 12 optical drives at maximum and records and reads. And supporting automatic verification after the file is recorded. Two verification algorithms, namely Message-Digest Algorithm 5 (MD5) and secure hash Algorithm 1 (Secure Hash Algorithm 1, SHA-1), are supported, and a verification code of each file is output to ensure the integrity of the recorded data. Automatic hierarchical management of data is supported. Any number of copy burns are supported. The optical disc is supported offline. High performance file querying. Optical disc storage encryption is supported. A standard file system interface is supported. The bandwidth is input to limit the speed. A variety of WORM modes support.

In one embodiment, system operation and maintenance management includes: the whole home page overview, the node management, the periodical detection and life cycle management of the optical disc library, the addition, modification and deletion operations of the optical disc library are supported, the operations of disc scanning, recording and reading of the optical disc library can be performed, and the state information of all optical discs can be checked.

Tape library function management may include function management based on a graphical user interface (Graphical User Interface, GUI) and function management based on a secondary development interface. The user (manager) can perform operations such as policy management, task management, data management, media management, user management and the like in a GUI interface operation mode; the secondary development interface, i.e., the API set, may provide a standard system call interface for tape library management for use by the management system. The management of multiple storage functions of the tape library can be realized through the two interfaces, and the specific functions can be as follows, including but not limited to the management of data, the management of storage strategies and tasks, the management of storage space, the management of media, the management of equipment and systems, the management of system expansion and development, etc. in classification: data life cycle management, task control, medium life cycle management, medium mixed loading management, expansion of tape library slots and the like. The functions managed in the above description are all common functions when the cloud-like tape library is used as a storage device, and are not described in detail herein.

Fig. 15 shows an electronic archive multimedia fusion storage management system provided in an embodiment of the present application, which includes an acquisition module 1, a service management module 2, a display module 3, and a storage module 4.

The acquisition module 1 is used for acquiring the management request input by the user.

The service management module 2 is configured to perform corresponding management operations on three storage devices, i.e., the distributed storage device, the optical disc library, and the tape library, and the electronic files stored on the storage devices according to the management request.

And a display module 3 for displaying the result of the management operation.

And the storage module 4 is used for storing the electronic files and comprises three storage devices, namely distributed storage, an optical disc library and a tape library.

The distributed storage uses a magnetic disk as a storage medium, the optical disc library uses an optical disc as a storage medium, and the magnetic tape library uses a magnetic tape as a storage medium. Management operations that may be performed by the service management module 2 include, but are not limited to, storage data visual management, storage device visual management, storage data inspection, and storage log management.

The acquisition module 1 and the display module 3 are positioned on the client layer in fig. 1, namely, each client side forming the client layer is provided with the acquisition module 1, and the acquisition module is used for acquiring a management request input by a user on the client side; meanwhile, each client is further provided with a display module 3, which is used for displaying the result of the management operation performed by the service management module 2 on the front-end interface for the user to review. The service management module 2 is located in the service layer in fig. 1, that is, a service management module 2 is disposed in a management server of the service layer, and is configured to perform corresponding management operation on data or equipment in the storage module 4 according to the management request acquired by the acquisition module 1, and feed back an operation result to the display module 3 of the client. The storage module 4 is located in the data layer in fig. 1, that is, a storage database of the data layer is provided with the storage module 4, and electronic archive data is stored therein, so as to set an interface for the service management module 2 to review or call the data and the device information stored thereon.

In one embodiment, the service management module 2 may include: a storage data visual management module 21, a storage device visual management module 22, a storage data inspection module 23, and a storage log management module 24.

The stored data visual management module 21 is configured to query the total data amount of the electronic files and the data amounts of various electronic files, search the corresponding electronic files according to any data information of the electronic files, obtain the storage device and other data information of the electronic files, and operate the electronic files.

The storage device visual management module 22 is configured to view the storage condition of each storage device and monitor the operation state of each storage device. The storage condition of the storage device may include total capacity, used capacity and storage duty ratio, and the running state of the storage device includes the health state and utilization rate of the CPU, the memory or the network card.

The storage data checking module 23 is configured to perform detection on different frequencies of electronic archive data of different importance levels stored on different storage devices according to a preset storage data checking policy;

the storage log management module 24 is configured to view, download, or export at least one of the storage logs.

In one embodiment, the visual management module 21 of storage data may query the electronic files stored on each storage device according to metadata information reported by the distributed storage, the optical disc library and the tape library.

In one embodiment, the storage device visual management module 22 may check capacity information of each storage device according to a data capacity query interface provided by the distributed storage, the optical disc library and the tape library, and may check and monitor an operation state of each storage device through a health state query interface and a real-time utilization rate query interface of a CPU, a health state query interface and a real-time utilization rate query interface of a memory, or a health state query interface and a real-time utilization rate query interface of a network card provided by the distributed storage, the optical disc library and the tape library.

In one embodiment, the storage data inspection module 23 may generate a data inspection task according to a storage data inspection policy and send the data inspection task to each storage device for automatic execution after receipt by the storage device.

In one embodiment, the storage log management module 24 may search, view, download, or export the corresponding storage log based on a user operation log access interface and a system operation log access interface provided by the distributed storage, optical disc library, and tape library.

In one embodiment, the service management module 2 may also include a storage device function management module. The storage device function management module is used for integrating and uniformly managing other functions of each storage device except the storage function.

In one embodiment, the distributed storage in the storage module 4 provides an NFS/CIFS/FTP data access interface, so that data can determine specific security access policies according to different media technical characteristics, and the expansibility of the system is improved.

In one embodiment, the optical disc library in the distributed storage 4 also provides an NFS/CIFS/FTP data access interface, which enables access to files of the optical disc library, supports a portable operating system interface (Portable Operating System Interface, POSIX) standard, and supports an NFS/CIFS/FTP network file access protocol.

In one embodiment, the disk library in the distributed storage 4 provides an API data access interface, so that tracking and management of the tape library, the storage medium and the drive are realized, and a user can discover abnormal equipment or medium in time through the improved API interface and perform relevant treatment according to the abnormal reminding.

It should be noted that, because the content of information interaction and execution process between the above units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other manners. For example, the apparatus/network device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In the description above, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims (10)

1. A management method for multi-medium fusion storage of electronic files is characterized by comprising the following steps:

acquiring a management request input by a user;

executing corresponding management operation on three storage devices, namely distributed storage, an optical disc library and a tape library, and electronic files stored on each storage device according to the management request; the distributed storage takes a magnetic disk as a storage medium, the optical disk library takes an optical disk as a storage medium, and the magnetic tape library takes a magnetic tape as a storage medium; the management operation comprises visual management of storage data, visual management of storage equipment, storage data checking and storage log management;

And displaying the result of the management operation on a front-end interface.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the visual management of the stored data comprises inquiring the total data volume of the electronic file and the data volumes of various electronic files, searching the electronic file according to any data information of the electronic file, obtaining storage equipment and other data information of the electronic file, and operating the electronic file;

the visual management of the storage equipment comprises checking the storage condition of each storage equipment and monitoring the operation state of each storage equipment, wherein the storage condition of the storage equipment comprises total capacity, used capacity and storage duty ratio, and the operation state of the storage equipment comprises the health state and the utilization rate of a CPU (central processing unit), a memory or a network card;

the stored data checking comprises the steps of detecting different frequencies of the electronic archive data of different importance levels stored on different storage devices according to a preset stored data checking strategy;

the storage log management includes at least one of viewing, downloading, or exporting storage logs.

3. The method of claim 2, wherein monitoring the operational status of each of the storage devices comprises:

Monitoring the health status of the CPU, the memory or the network card of each storage device, and displaying an alarm prompt on a front-end interface when the CPU, the memory or the network card fails;

setting the utilization rate threshold of the CPU, the memory or the network card of each storage device according to the characteristics of each storage device, and displaying an alarm prompt on a front-end interface when the utilization rate of the CPU, the memory or the network card is monitored to exceed the utilization rate threshold.

4. The method according to claim 2, characterized in that in performing said visual management of stored data, it comprises the steps of:

and inquiring the electronic files stored on each storage device through the distributed storage and the metadata information reported by the optical disc library and the magnetic tape library.

5. The method of claim 2, wherein when performing the visual management of the storage device, comprising the steps of:

checking capacity information of each storage device through a data capacity inquiry interface provided by the distributed storage, the optical disc library and the magnetic tape library;

and checking and monitoring the running state of each storage device through the health state query interface and the real-time utilization rate query interface of the CPU, the health state query interface and the real-time utilization rate query interface of the memory, or the health state query interface and the real-time utilization rate query interface of the network card, which are provided by the distributed storage, the optical disc library and the magnetic tape library, so as to realize the visual management of the storage devices.

6. The method according to claim 2, characterized in that in performing the stored data check, it comprises the steps of:

and generating a data checking task according to the stored data checking strategy, and receiving the data checking task by the distributed storage, the optical disc library and the tape library and automatically executing the data checking task, thereby realizing the stored data detection.

7. The method of claim 2, wherein when performing the storage log management, comprising the steps of:

and searching, checking, downloading or exporting the user operation log and the system operation log of each storage device through the distributed storage, the optical disc library and the log access interface provided by the tape library, so as to realize the storage log management.

8. The method of claim 1, wherein the managing operation further comprises storage device function management, wherein the storage device function management is unified management of storage functions of each of the three types of storage devices.

9. A management system for multi-media fusion storage of electronic files, the system comprising:

the acquisition module is used for acquiring a management request input by a user;

The service management module is used for executing corresponding management operation on three storage devices of distributed storage, an optical disc library and a tape library and the electronic files stored on each storage device according to the management request; the distributed storage takes a magnetic disk as a storage medium, the optical disk library takes an optical disk as a storage medium, and the magnetic tape library takes a magnetic tape as a storage medium; the management operation comprises visual management of storage data, visual management of storage equipment, storage data checking and storage log management;

the display module is used for displaying the result of executing the management operation;

and the storage module is used for storing the electronic files and comprises three storage devices, namely the distributed storage, the optical disc library and the tape library.

10. The system of claim 9, wherein the service management module comprises:

the storage data visual management module is used for inquiring the total data volume of the electronic file and the data volume of various electronic files, searching the electronic file according to any data information of the electronic file, obtaining storage equipment and other data information of the electronic file, and operating the electronic file;

The visual management module of the storage equipment is used for checking the storage condition of each storage equipment and monitoring the operation state of each storage equipment, wherein the storage condition of the storage equipment comprises total capacity, used capacity and storage duty ratio, and the operation state of the storage equipment comprises the health state and the utilization rate of a CPU (central processing unit), a memory or a network card;

the storage data checking module is used for detecting different frequencies of the electronic archive data of different importance levels stored on different storage devices according to a preset storage data checking strategy;

and the storage log management module is used for at least one of checking, downloading or exporting the storage log.

Images