CN112379826A - Application method of storage integration technology - Google Patents

Abstract

An application method of storage integration technology. The invention relates to the field of computer data storage, realizes the function of data storage integration by dividing luns and applying deduplication, and allows a storage management programmer to improve the use and control capacity of enterprise storage resources by carrying out corresponding storage integration on data so as to meet the continuously increasing storage requirements of enterprises.

Description

Application method of storage integration technology

Technical Field

The invention relates to an application method of a storage integration technology, belonging to the field of computer data security processing.

Background

With the advent of the big data age, data has attracted more and more attention as the most important asset of enterprises. The more centralized the data, the greater the security risk, and the ever increasing amount of data makes the dependence on storage stronger and stronger, so the storage of data is an important link.

The need for storage consolidation stems from the fact that: that is, storage requirements are increasing. Within some enterprises, this demand is growing almost one hundred percent per year.

Most enterprises strive to meet the ever-increasing storage demands from users and applications by continually adding disks, arrays, and servers. Over time, these enterprises have faced the problem of storage decentralization, with data stored on disks and systems in data centers, throughout the enterprise, and even throughout the world through remote offices and mobile users. The high-speed incremental storage of assets presents the following four problems:

the cost of the new storage increases;

overhead increase of storage management;

potential security threats present on one or more storage platforms

Excess capacity of the equipment (e.g. energy and cooling)

Although the using method can meet the requirement of the servers for quickly reading and writing data, the using method cannot meet the requirements of sharing and uniformly managing storage resources, and the requirements that one server is down and another server cannot take over the ongoing tasks in real time. In this case, it is conventional to have a server in the environment use a network file system protocol, such as: NFS (Linux/UNIX) or cifs (windows) shares its own file system for access by other servers, users or applications. The method solves the problem of storage integration, but the use scheme is difficult to meet the requirements when the storage capacity is large and the application occasions with low delay and high IO throughput are needed.

In addition, the needs for heterogeneous multi-operating system platform support and scalability of system capacity and performance are increasingly prominent for current application environments, and especially in media, government, scientific and large engineering, data center, etc. environments, where Windows and Linux are used in a mixed manner, and the increase of storage capacity and IO performance has become a necessary requirement year by year.

In conventional storage systems, users often deploy sufficient physical storage space beyond the actual demand in order to ensure that storage capacity is adequately utilized. In actual use, however, deployment capacity is often underutilized. Industry research organizations have found that in some projects, the actual usage capacity is only 20% -30% of the deployed capacity. Therefore, the method of the storage consolidation technology is produced, and aims to realize higher storage capacity utilization rate and bring greater return on investment.

Because of this, storage consolidation has become a preferred choice for many data centers.

Disclosure of Invention

The invention aims to realize the integration function of data storage, a series of processing is carried out on data to store the data, so that the data and functions are completely consistent between two storages, when one storage is down, the other storage can automatically access the data in real time and execute all tasks executed by the damaged storage, and thus, the safe last line of defense backup can be realized and the loss caused by the loss of user data is prevented.

The technical scheme of the invention realizes an integration method based on storage, and the method comprises the following steps:

firstly: the user's applications run on the application server AS, which access data on the storage device via the file access interface.

Secondly, the method comprises the following steps: the application server AS accesses the storage device to obtain high-performance data input and output access, and processes the request of reading and writing user data, and the data access does not need to pass through the metadata server. In the whole file access process, an application server AS (client) firstly accesses a metadata server to obtain a file logic position, then accesses the metadata server to obtain a file layout, and finally accesses a storage physical position to obtain data.

Drawings

FIG. 1 is a flow chart for implementing data storage consolidation

Detailed Description

Overall scheme design

The NetApp FAS storage system can simultaneously support access modes of various storage architectures such as FC SAN, iSCSI, NAS and the like on the same storage platform, thereby facilitating centralized management of data of a business system by enterprise users, simplifying management and facilitating maintenance.

Here the storage is integrated in iSCSI fashion. iSCSI is an IP-based network storage technology, and can reduce the total ownership cost and investment cost of a system on the premise of ensuring high performance. Considering the situation of the existing service systems windows2000 and windows2003 server hosts, the data of the existing service systems are stored and integrated in a high-performance storage system in a centralized manner, the data of the existing service systems are respectively migrated to new storage system platforms, all the existing operating system platforms, database types and storage management modes are continuously reserved, and the main advantage after the iSCSI is adopted is that a block-level protocol is adopted, so that the compatibility of the application is very good. Compared with FC SAN, the compatibility of iSCSI protocol itself is very good, there is no complex compatibility problem in FC SAN, and the maintenance cost is greatly reduced. NAS and iSCSI are based on IP network storage technology, can reduce the total cost of ownership and investment cost of the system on the premise of guaranteeing the high performance, can fully meet the data storage requirement of the business system through the integrated storage.

The configuration list is as follows:

1. the FAS2020A is provided with a double controller, 2GB Cache, 4Gb optical fiber ports and 4 1Gb Ethernet ports;

2. FCP, iSCSI, CIFS (optional) and other protocols, and simultaneously supports FC SAN, IP SAN, NAS (windows platform file sharing) and other storage modes;

3. according to the current requirement, the storage capacity is firstly configured with 8 blocks of 300GB 15000-SAS disks, and then the capacity can be expanded online according to the data growth condition;

4. configuring storage Cluster Cluster, Snapshot Snapshot software and Dedup (A-SiS) repeated data deletion software to realize high availability of storage access and data Snapshot backup;

volume group capacity and RAID partitioning

For performance, the 15000 to 300GB SAS hard disk is fully configured, and 12 disks may be configured first in consideration of the need for future capacity expansion. Because the number of the engineering disks is small, the RAID mode recommends using two RAID4(5D +1P + 1S); with the expansion of capacity, the number of the disks is increased, RAID-DP can be adopted, and very high data reliability can be realized with lower cost. The hot spare disk of the storage device is global and is not limited by the slot position, so that the corresponding hot spare disk only needs to be reserved according to the proportion. Hot plates in proportions of 2 to 3% are generally preferred. However, because the data volume of the project is small, at least 2 disks are suggested to be reserved as hot spare disks under the safety consideration. (at least 1 hot spare plate for each controller)

Because most databases integrated in the project are small-capacity databases, in order to avoid the performance problem of the small-capacity databases during integration, 1 Aggregate is firstly created, each controller has 1, each Aggregate is allocated with 6 disks (minus 1 hot spare disk), then FlexVol is allocated in the Aggregate disks, only Data files or Log files of 1 service system are placed on each FlexVol, and Control files and Binary files of all service system databases can be uniformly placed on 1 FlexVol. The benefit of this is that all databases share the performance of the Aggregate disk, avoiding the performance bottleneck of the hot spot disk.

Capacity adjustment for iSCSI

The storage equipment has a complete unified network storage function and can simultaneously support FC SAN on the same equipment; NFS; iSCSI; CIFS, and the like. And the capacity allocation proportion of the NAS and the SAN in the same equipment can be flexibly adjusted. The specific adjustment method is as follows:

1) freeing unused storage space (through FlexVol's reduced volume space setting, or destroying unused system volume data)

2) Allocating the newly allocated space to a new volume

3) The design related to FC/iSCSI LUN or NFS export ip network can be created in a new volume since this solution uses a network storage system supporting nas (ip san), a high-performance gigabit ethernet connection needs to be used between the client and the FAS storage system. For reliability reasons, redundant network connection designs need to be employed. It is proposed to use two gigabit switches dedicated to the connection between the client and the NetApp FAS storage system.

The 2 GE gigabit interfaces of each storage controller in the FAS2020A device are connected to two gigabit switches, respectively, using Single-Mode Ethernet Channel technology in VIF (virtual port/trunk aggregation) as 1 master 1 backup. Of course, the following network connection methods may also be employed:

the 2 GE gigabit interfaces of each storage controller in the FAS2020A device are connected to the same high performance core gigabit switch using Multi-Mode Ethernet Channel technology in VIF (virtual port/trunk aggregation) as a redundant load balancing approach.

If the active GE gigabit port of the FAS storage system fails, the standby port can be automatically switched to. It is noted here that in some complex network environments, switches need to enable the spanning tree STP protocol in order to avoid loop formation. When a port switch occurs at this time, since the switch needs to recalculate the Spanning Tree, it takes a long time (default about 50 seconds) to complete convergence if the normal STP protocol is used. It is proposed to use the PortFast configuration of a gigabit switch on the ports connected to the FAS storage system and the front-end switch, where the switch ports can complete the switch immediately upon a change of user port state (directly into the forwarding state) without the need to complete the switch via the STP protocol.

Storage virtualization management

The FlexVol technology not only greatly improves the performance of data access by increasing the dispersion degree of the bottom layer IO, but also supports real sharing and dispatching of different applications on a storage space by the unique technology of the storage sharing pool, greatly improves the utilization rate of the storage space, and simplifies the complexity of storage space management. However, in a general management method for pre-allocating storage space, a certain space is allocated to each application system based on a space requirement provided by a service manager, the space allocation is fixed, the space allocated to each application is an actual space with a given size, regardless of the size of the actual space used by the application, the space allocated to each application system occupies the physical space allocated to the application system, and re-adjusting the space is a very time-consuming operation and affects the normal operation of the application system in the period of time. Therefore, the conventional method often causes some applications of users to be expanded due to the small pre-allocated space, and meanwhile, a large amount of free space in the application is unused, but the applications which need to be expanded cannot be adjusted. The only method for users to solve the problem is to continuously purchase capacity, so that a vicious circle is entered that the larger the capacity of a system is, the larger the space waste is, the more frequent capacity expansion is needed, and finally, the low utilization rate of the storage space of the users and the complex management of the storage space are caused. While the advanced storage space management and usage strategy provided by FlexVol completely avoids these problems.

System extensibility

If higher speed and capacity storage is needed in the future, disk shelves in FAS2020A may be used in higher-end 2050A/3140A series storage devices to achieve investment protection. When the machine head of the system is replaced, the whole disk array does not need to carry out data migration, and can be immediately accessed into a new machine head to complete expansion.

When the system increases the disks, the reconstruction can be performed online, the reconstruction time of the RAID is very short (the reconstruction time of adding 4 disks is about 1 second), and therefore the influence on the performance of the system can be ignored. However, it should be noted that in the SAN environment, the host side also needs to perform LUN expansion, partitioning, and formatting related operations.

Claims (4)

1. An application method of a storage integration technology is characterized in that:

(1) the application program of the user runs on an application server AS;

(2) an application server AS accesses a metadata server to obtain a file logic position;

(3) an application server AS accesses a metadata server to obtain a file layout;

(4) the application server accesses the storage physical position to acquire data;

(5) the application program directly accesses the data on the storage device through the file access interface.

2. A method (2) AS claimed in claim 1 wherein the file logical location is marked by accessing the metadata server through the AS.

3. The method (3) of claim 1 wherein accessing the metadata server via the AS comprises obtaining an overall layout of the file.

4. The method (5) as claimed in claim 1 wherein said accessing by file comprises: access file logical locations, access file layouts, access storage physical locations.

Images