WO2021098267A1

WO2021098267A1 - Magnetic disk processing method, system, and device, and readable storage medium

Info

Publication number: WO2021098267A1
Application number: PCT/CN2020/104004
Authority: WO
Inventors: 李宏伟
Original assignee: 苏州浪潮智能科技有限公司
Priority date: 2019-11-22
Filing date: 2020-07-24
Publication date: 2021-05-27
Also published as: CN110908609A; CN110908609B

Abstract

A magnetic disk processing method, system, and device, and a readable storage medium, the method comprising: acquiring a magnetic disk processing command; when the magnetic disk processing command is a magnetic disk adding command, processing a target magnetic disk as a raw device magnetic disk; dividing the storage space of the raw device magnetic disk into hidden space and data space; mapping the data space to an image device, and generating an access path for the image device, such that a virtual machine accesses the image device by means of the access path. The present application enables the raw device magnetic disk to store virtual machine configuration data, and thereby support the high availability of the virtual machine; in addition, by means of mapping the data space to an image device and generating an access path for the image device, such that the virtual machine can access the image device by means of the access path, the reliability of the system is increased whilst the IO performance of the virtual machine is improved.

Description

Method, system, equipment and readable storage medium for disk processing

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on November 22, 2019, the application number is 201911157820.0, and the invention title is "a method, system, equipment and readable storage medium for disk processing", all of which The content is incorporated in this application by reference.

Technical field

This application relates to the field of disk processing, and in particular to a method, system, device, and readable storage medium for disk processing.

Background technique

In the field of server virtualization, in order to ensure the uninterrupted operation of enterprise-level services, High Available (HA) virtual machines can quickly switch the virtual machine to another host and keep the business running after the host where the user virtual machine is located fails. Minimize the interruption time. The commonly used virtual machine HA solution is to store the virtual disks of the virtual machines in the shared storage pool in the form of files. When a host in the cluster fails, the virtualization management software quickly selects a host in the cluster corresponding to the shared storage pool lock. And pull up all the problematic virtual machines on it. A premise here is that the shared storage pool is a container that can store virtual machine disk image files and configuration files, so that all hosts can share access and start virtual machines.

The bottom layer of the shared storage pool is generally implemented by a cluster file system or NFS. Its advantage is that it can store heterogeneous virtual machine data, is easy to use, and can ensure the consistency of virtual machine files at the file system level; but its disadvantages are also obvious. The introduction of the file system abstraction layer allows the IO of the virtual machine to pass through the file system layer before it can fall into the underlying block device, which inevitably brings performance loss, which is unacceptable for application scenarios with heavy IO load.

In order to improve the IO performance of the virtual machine, a common practice in the industry is to remove the file system layer and directly allow the virtual machine to directly access the raw device disk. However, raw device disks cannot store metadata such as virtual machine configuration files and cannot support virtual machine HA. Although the performance is improved, the reliability of the system is reduced.

Therefore, how to improve the IO performance of the virtual machine while improving the reliability of the system is a technical problem that needs to be solved by those skilled in the art.

Summary of the invention

The purpose of this application is to provide a method, system, device, and readable storage medium for disk processing, which are used to improve the IO performance of a virtual machine while improving the reliability of the system.

In order to solve the above technical problems, this application provides a method for disk processing, which includes:

Obtain disk processing commands;

When the disk processing command is a disk add command, the target disk is processed as a raw device disk;

Dividing the storage space of the bare device disk into a hidden space and a data space; wherein the hidden space is used to store virtual machine configuration data;

The data space is mapped to the imaging device, and an access path is generated for the imaging device, so that the virtual machine can access the imaging device through the access path.

Optionally, it is characterized in that it further includes:

When detecting that there is a faulty host, pull up all faulty virtual machines on the faulty host;

Determine the target hidden space corresponding to the failed virtual machine, and obtain virtual machine configuration data in the target hidden space;

Select a host from the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data.

Optionally, the processing the target disk as a raw device disk includes:

The target disk is formatted, and a preset magic number is added to the header of the target disk to obtain the raw disk device.

Optionally, when the disk processing command is a disk delete command, it further includes:

Determine the disk to be deleted according to the disk delete command;

The imaging device and access path corresponding to the disk to be deleted are deleted.

This application also provides a disk processing system, which includes:

Obtaining module for obtaining disk processing commands;

A processing module, which is used to process the target disk as a raw device disk when the disk processing command is a disk add command;

The dividing module is used to divide the storage space of the bare device disk into a hidden space and a data space; wherein the hidden space is used to store virtual machine configuration data;

The mapping module is configured to map the data space into an imaging device and generate an access path for the imaging device, so that the virtual machine can access the imaging device through the access path.

Optionally, it also includes:

The pull-up module is used to pull up all the faulty virtual machines on the faulty host when a faulty host is detected;

The first determining module is configured to determine the target hidden space corresponding to the faulty virtual machine, and obtain virtual machine configuration data in the target hidden space;

The startup module is used to select a host in the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data.

Optionally, the processing module includes:

The processing sub-module is used to format the target disk and add a preset magic number to the header of the target disk to obtain the raw disk device.

Optionally, it also includes:

The second determining module is configured to determine the disk to be deleted according to the disk delete command when the disk processing command is a disk delete command;

The deleting module is used to delete the imaging device and the access path corresponding to the disk to be deleted.

This application also provides a disk processing device, which includes:

Memory, used to store computer programs;

The processor is used to implement the steps of any one of the above-mentioned disk processing methods when the computer program is executed.

The present application also provides a readable storage medium having a computer program stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the disk processing method described in any one of the above are implemented.

The method for disk processing provided by this application includes: obtaining a disk processing command; when the disk processing command is a disk adding command, processing the target disk as a raw device disk; dividing the storage space of the raw device disk into hidden space and data space; Among them, the hidden space is used to save the virtual machine configuration data; the data space is mapped to the imaging device, and an access path is generated for the imaging device, so that the virtual machine can access the imaging device through the access path.

The technical solution provided by this application treats the target disk as a raw device disk, and then divides the storage space of the raw device disk into hidden space and data space, so that the bare device disk can store virtual machine configuration data, thereby supporting virtual The high availability of the machine; at the same time, by mapping the data space to the imaging device and generating an access path for the imaging device, the virtual machine can access the imaging device through the access path, which improves the IO performance of the virtual machine while improving the system’s performance reliability. This application also provides a disk processing system, device and readable storage medium, which have the above-mentioned beneficial effects, and will not be repeated here.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

FIG. 1 is a flowchart of a method for disk processing provided by an embodiment of the application;

FIG. 2 is a flowchart of another method for processing a disk provided by an embodiment of the application;

FIG. 3 is a structural diagram of a disk processing system provided by an embodiment of the application;

4 is a structural diagram of another disk processing system provided by an embodiment of the application;

FIG. 5 is a structural diagram of a disk processing device provided by an embodiment of the application.

Detailed ways

The core of this application is to provide a method, system, device, and readable storage medium for disk processing, which are used to improve the IO performance of the virtual machine while enhancing the reliability of the system.

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Please refer to FIG. 1. FIG. 1 is a flowchart of a disk processing method provided by an embodiment of the application.

It specifically includes the following steps:

S101: Obtain a disk processing command;

Based on the prior art, in order to improve the IO performance of the virtual machine, the file system layer is removed, and the virtual machine directly accesses the raw device disk. However, raw device disks cannot store metadata such as virtual machine configuration files and cannot support virtual machine HA, which improves performance but reduces system reliability; therefore, this application provides a disk processing method to solve the above problems ；

Optionally, the disk processing commands mentioned here may include, but are not limited to, disk add commands, disk delete commands, or disk change commands, among which:

When the disk processing command is a disk delete command, you can also determine the disk to be deleted according to the disk delete command, and delete the corresponding imaging device and access path of the disk to be deleted, so that the user can eliminate the faulty hard disk or the old hard disk in the system ；

When the disk processing command is a disk change command, the disk to be changed can also be determined according to the disk change command, and the disk to be changed can be changed to an imaging device. Specifically, the following steps can be performed:

The storage space of the disk to be changed is divided into hidden space and data space, and the data space is mapped to the imaging device, and an access path is generated for the imaging device, so that the virtual machine can access the imaging device through the access path, so that the user needs to access the imaging device through the access path. Add a new disk to the system, but directly process the original disk to get the image device that the virtual machine can access.

In a specific embodiment, a new udev rule can be added to the host to obtain disk processing commands, so that the host can perceive the addition, change, and deletion of disk devices, and call the rawdisk_manager tool to perform processing on the original disk. To process.

S102: When the disk processing command is a disk adding command, process the target disk as a raw device disk;

The target disk mentioned here is the disk newly added to the system, and the target disk needs to be processed as an imaging device that the virtual machine can access;

Raw device (raw device), also known as raw partition (raw partition), is a special block device file that is not read by Unix through the file system. The application is responsible for reading and writing operations on it, without going through the file system. Buffer, it is a device that is not directly managed by the operating system. This kind of device lacks the operating system layer and has higher I/O efficiency. The raw device disk mentioned here is the disk that is not read by Unix through the file system;

Optionally, as mentioned here, the target disk is treated as a raw device disk, which can be specifically:

Format the target disk, and add a preset magic number to the head of the target disk to obtain a raw disk device.

S103: Divide the storage space of the raw device disk into a hidden space and a data space;

The hidden space mentioned here is used to store virtual machine configuration data. Based on the fact that existing raw device disks cannot store metadata such as virtual machine configuration files, this application creatively divides the storage space of the bare device disk into hidden space and data space. And save the virtual machine configuration data in the hidden space, so that the raw device disk supports the high availability of the virtual machine.

S104: Map the data space to the imaging device, and generate an access path for the imaging device, so that the virtual machine can access the imaging device through the access path.

In a specific embodiment, the raw device disk can be divided into two parts, the front 32M space and the remaining space; the /dev/disk/by-id/rawdisk-<scsi id> path is generated for the remaining space for virtual machine access , And the first 32M space is a hidden space, used to retain metadata such as virtual machine configuration data.

Based on the above technical solution, a disk processing method provided by this application is to process the target disk as a raw device disk, and then divide the storage space of the raw device disk into hidden space and data space, so that the raw device disk can store The virtual machine configuration data can support the high availability of the virtual machine. At the same time, by mapping the data space to the imaging device, and generating an access path for the imaging device, the virtual machine can access the imaging device through the access path, which improves the virtual The machine IO performance improves the reliability of the system at the same time.

Based on the above-mentioned embodiments, the present application also provides another method for disk processing, which is described below with reference to FIG. 2.

Please refer to FIG. 2, which is a flowchart of another disk processing method provided by an embodiment of the application.

It specifically includes the following steps:

S201: When detecting that there is a faulty host, pull up all faulty virtual machines on the faulty host;

S202: Determine the target hidden space corresponding to the faulty virtual machine, and obtain virtual machine configuration data in the target hidden space;

S203: Select a host in the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data.

Based on the above technical solution, when a faulty host is detected, the embodiment of the present application pulls up all faulty virtual machines on the faulty host, then determines the target hidden space corresponding to the faulty virtual machine, and obtains the virtual machine in the target hidden space Configure the data, and finally select a host in the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data, thereby realizing the high availability of the virtual machine.

Please refer to FIG. 3, which is a structural diagram of a disk processing system provided by an embodiment of the application.

The system can include:

The obtaining module 100 is used to obtain disk processing commands;

The processing module 200 is used to process the target disk as a raw device disk when the disk processing command is a disk add command;

The dividing module 300 is used to divide the storage space of the bare device disk into a hidden space and a data space; among them, the hidden space is used to store virtual machine configuration data;

The mapping module 400 is used to map the data space to the imaging device and generate an access path for the imaging device, so that the virtual machine can access the imaging device through the access path.

Please refer to FIG. 4, which is a structural diagram of another disk processing system provided by an embodiment of the application.

The system can also include:

The first determining module is used to determine the target hidden space corresponding to the faulty virtual machine, and obtain virtual machine configuration data in the target hidden space;

The processing module 200 may include:

The processing sub-module is used to format the target disk and add a preset magic number to the head of the target disk to obtain a raw disk device.

The system can also include:

The second determining module is used to determine the disk to be deleted according to the disk delete command;

The delete module is used to delete the imaging device and access path corresponding to the disk to be deleted.

Each component of the above system can be actually applied to the following embodiments:

The acquisition module acquires the disk processing command; when the disk processing command is a disk add command, the processing module processes the target disk as a raw device disk; the dividing module divides the storage space of the raw device disk into hidden space and data space; the mapping module divides the data space Map as an imaging device, and generate an access path for the imaging device, so that the virtual machine can access the imaging device through the access path.

When a faulty host is detected, the pull-up module pulls up all the faulty virtual machines on the faulty host; the first determination module determines the target hidden space corresponding to the faulty virtual machine, and obtains the virtual machine configuration data in the target hidden space; start The module selects a host in the cluster corresponding to the shared storage pool as the target host, and starts the corresponding failed virtual machine on the target host according to the virtual machine configuration data.

When the disk processing command is a disk delete command, the second determination module determines the disk to be deleted according to the disk delete command; the delete module deletes the imaging device and the access path corresponding to the disk to be deleted.

Please refer to FIG. 5, which is a structural diagram of a disk processing device provided by an embodiment of the application.

The disk processing device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 522 (for example, one or more processors) and a memory 532, one or more More than one storage medium 530 for storing application programs 542 or data 544 (for example, one or one storage device with a large amount of storage). Among them, the memory 532 and the storage medium 530 may be short-term storage or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the device. Furthermore, the processor 522 may be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the disk processing device 500.

The disk processing device 500 may also include one or more power supplies 525, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, and/or one or more operating systems 541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The steps in the disk processing method described in FIGS. 1 to 2 are implemented by the disk processing device based on the structure shown in FIG. 5.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described system, device, and module can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed devices, equipment, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of modules is only a logical function division, and there may be other divisions in actual implementation, for example, multiple modules or components can be combined or integrated To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or modules, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software function modules.

If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which can be a personal computer, a function calling device, or a network device, etc.) execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

The method, system, device, and readable storage medium for disk processing provided by this application are described in detail above. Specific examples are used in this article to illustrate the principles and implementation of the application. The description of the above examples is only used to help understand the methods and core ideas of the application. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations. There is any such actual relationship or sequence between operations. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. Without more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

Claims

A method for disk processing, characterized in that it comprises:

Obtain disk processing commands;

When the disk processing command is a disk add command, the target disk is processed as a raw device disk;

Dividing the storage space of the bare device disk into a hidden space and a data space; wherein the hidden space is used to store virtual machine configuration data;

The data space is mapped to the imaging device, and an access path is generated for the imaging device, so that the virtual machine can access the imaging device through the access path.
The method according to claim 1, further comprising:

When detecting that there is a faulty host, pull up all faulty virtual machines on the faulty host;

Determine the target hidden space corresponding to the failed virtual machine, and obtain virtual machine configuration data in the target hidden space;

Select a host from the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data.
The method according to claim 1, wherein the processing the target disk as a raw device disk comprises:

The target disk is formatted, and a preset magic number is added to the header of the target disk to obtain the raw disk device.
The method according to claim 1, wherein when the disk processing command is a disk delete command, the method further comprises:

Determine the disk to be deleted according to the disk delete command;

The imaging device and access path corresponding to the disk to be deleted are deleted.
A disk processing system, which is characterized in that it comprises:

Obtaining module for obtaining disk processing commands;

A processing module, which is used to process the target disk as a raw device disk when the disk processing command is a disk add command;

The dividing module is used to divide the storage space of the bare device disk into a hidden space and a data space; wherein the hidden space is used to store virtual machine configuration data;

The mapping module is configured to map the data space into an imaging device and generate an access path for the imaging device, so that the virtual machine can access the imaging device through the access path.
The system according to claim 5, further comprising:

The pull-up module is used to pull up all the faulty virtual machines on the faulty host when a faulty host is detected;

The first determining module is configured to determine the target hidden space corresponding to the faulty virtual machine, and obtain virtual machine configuration data in the target hidden space;

The startup module is used to select a host in the cluster corresponding to the shared storage pool as the target host, and start the corresponding failed virtual machine on the target host according to the virtual machine configuration data.
The system according to claim 5, wherein the processing module comprises:

The processing sub-module is used to format the target disk and add a preset magic number to the header of the target disk to obtain the raw disk device.
The system according to claim 5, further comprising:

The second determining module is configured to determine the disk to be deleted according to the disk delete command when the disk processing command is a disk delete command;

The deleting module is used to delete the imaging device and the access path corresponding to the disk to be deleted.
A disk processing device, characterized in that it comprises:

Memory, used to store computer programs;

The processor is configured to implement the steps of the disk processing method according to any one of claims 1 to 4 when the computer program is executed.
A readable storage medium, characterized in that a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the method for disk processing according to any one of claims 1 to 4 are realized .