CN109558205B - Disk access method and device - Google Patents

Disk access method and device Download PDF

Info

Publication number
CN109558205B
CN109558205B CN201710875562.4A CN201710875562A CN109558205B CN 109558205 B CN109558205 B CN 109558205B CN 201710875562 A CN201710875562 A CN 201710875562A CN 109558205 B CN109558205 B CN 109558205B
Authority
CN
China
Prior art keywords
lock
virtual machine
disk
host
predetermined
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710875562.4A
Other languages
Chinese (zh)
Other versions
CN109558205A (en
Inventor
肖立琨
李翌
刘建军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing ZTE New Software Co Ltd
Original Assignee
Nanjing ZTE New Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing ZTE New Software Co Ltd filed Critical Nanjing ZTE New Software Co Ltd
Priority to CN201710875562.4A priority Critical patent/CN109558205B/en
Publication of CN109558205A publication Critical patent/CN109558205A/en
Application granted granted Critical
Publication of CN109558205B publication Critical patent/CN109558205B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/526Mutual exclusion algorithms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45583Memory management, e.g. access or allocation

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a disk access method and a device, wherein the method comprises the following steps: after the virtual machine accesses the predetermined disk, controlling the virtual machine to release a lock corresponding to the predetermined disk and caching the lock in a local memory of a host where the virtual machine is located; under the condition of receiving an access request of a virtual machine to a predetermined disk, acquiring a lock in a local memory; and controlling the virtual machine to access a predetermined disk. The invention solves the problems of low lock request efficiency and low disk write efficiency caused by a large number of lock requests when the virtual machine reads and writes the disk in the related technology.

Description

Disk access method and device
Technical Field
The invention relates to the field of communication, in particular to a disk access method and device.
Background
With the progress and development of virtualization technology, more and more enterprises, institutions and schools abandon the desktop scheme which is cumbersome and difficult to manage in the past when building own office and teaching environments. The technical scheme of virtualization which is more convenient, easy to use and safe is adopted. In these application scenarios, the solution of storage is crucial, and its efficiency, sharing, and security directly determine the performance of virtualization technology to the upper layer users. The currently popular network cloud storage scheme is not applicable to the above scenario due to latency, security (non-localization). The distributed storage scheme formed by multiple storage nodes is difficult to embody the characteristic of high availability in the application scene with small scale. While the cluster file system based on shared storage cannot be compared with the two previous storage schemes in large or ultra-large application scenarios, the cluster file system is more suitable for the above-mentioned small and medium-scale application scenarios. How to guarantee atomic access to the file system metadata is a heavy center in the overall storage scheme.
Fig. 1 is a schematic diagram of a virtualization cluster in the related art, as shown in fig. 1, hosts of a cluster file system are in the same local area network, and the hosts are in network intercommunication and share access to one or more storage devices. In an intranet, the response delay of the network is usually within 1ms, while the stored response speed is in the order of several ms. In addition, in order to ensure the atomicity of reading and writing of the lock itself, exclusive access needs to be performed on the disk where the lock is located, and this behavior will result in that all write operations to the disk are unavailable. That is, after modifying the implementation of the lock from network to storage, the greatest impact is the request efficiency of the lock and the write efficiency of the disk. To address the above deficiencies, the number of lock requests is reduced as much as possible.
FIG. 2 is a diagram illustrating a large number of locks associated with files in a cluster file system in the related art, and as shown in FIG. 2, a lock associated with a file is generally composed of an inode and a plurality of resource group locks. The usage scenario of the file inode lock cannot be reduced, but due to the particularity of the virtual machine using the file (see the foregoing description), the conflict of the file inode is determined to be relatively less in the usage scenario of the virtual machine. FIG. 3 is a diagram illustrating a file locking process in a cluster file system according to the related art, and as shown in FIG. 3, a process of requesting a lock during a standard write operation of a file is described. In the writing process, the file is also added with the association with the transaction lock. The file request transaction lock is used for recording a log, and the log is used for recovery once a host computer fails; requesting an inode lock to avoid other contexts from operating the file at the same time; and requesting a resource group lock for adding an available data segment for the file.
In the related art, when the virtual machine performs disk reading and writing, multiple pages are usually submitted at one time, and the lock in the above process is repeatedly requested and released. Not only a large number of Small Computer System Interface (SCSI) lock requests are generated, but also disk Input/Output (I/O) is generated, and these operations are meaningless in the usage scenario of the virtual machine, and the lock request efficiency and the disk write efficiency are also reduced.
Disclosure of Invention
The embodiment of the invention provides a disk access method and a disk access device, which are used for at least solving the problems of low lock request efficiency and low disk write efficiency caused by a large number of lock requests when a virtual machine reads and writes a disk in the related technology.
According to an embodiment of the present invention, there is provided a disk access method including: after the virtual machine accesses the predetermined disk, controlling the virtual machine to release a lock corresponding to the predetermined disk and caching the lock in a local memory of a host where the virtual machine is located; under the condition of receiving the access request of the virtual machine to the preset disk, acquiring the lock in the local memory; and controlling the virtual machine to access the preset disk.
Optionally, before controlling the virtual machine to access the predetermined disk, the method further includes: and establishing a data mapping with a preset size in a preset file system for the metadata and the data of the file corresponding to the preset disk.
Optionally, after controlling the virtual machine to release the lock corresponding to the predetermined disk, the method further includes: in the event that it is determined that the lock has been cached in the local memory for more than a predetermined length of time, the host releases the lock and updates an identifier indicating whether the lock was released by the host.
Optionally, before updating the identifier indicating whether the lock is released by the host, the method further includes: the identification is stored using predetermined bits on a predetermined disk.
Optionally, in a case that an access request of the virtual machine to the predetermined disk is received, acquiring the lock in the local memory includes: and acquiring the lock in the local memory under the condition of receiving the access request of the virtual machine to the preset disk within preset time.
According to another embodiment of the present invention, there is provided a disk access apparatus including: the cache module is used for controlling the virtual machine to release a lock corresponding to the preset disk and cache the lock in a local memory of a host where the virtual machine is located after the virtual machine accesses the preset disk; an obtaining module, configured to obtain the lock in the local memory when receiving an access request of the virtual machine to the predetermined disk; and the control module is used for controlling the virtual machine to access the preset disk.
Optionally, the control module is further configured to establish a data map of a predetermined size in a predetermined file system for metadata and data of a file corresponding to the predetermined disk before controlling the virtual machine to access the predetermined disk.
Optionally, the cache module is further configured to, after controlling the virtual machine to release the lock, if it is determined that the time for which the lock is cached in the local memory exceeds a predetermined time, the host releases the lock and updates the identifier indicating whether the lock is released by the host.
According to yet another embodiment of the present invention, there is also provided a storage medium including a stored program, wherein the program performs any one of the above methods when executed.
According to yet another embodiment of the present invention, there is also provided a processor for executing a program, wherein the program executes to perform the method of any one of the above.
According to the invention, after the virtual machine accesses the predetermined disk, the lock is only released in the host memory, and the disk can not be truly landed, so that when the subsequent virtual machine reads and writes the disk, the lock is only detected in the host memory, the virtual machine can access the predetermined disk, the SCSI request is not required, and the lock request frequency is reduced. Therefore, the problems that when the virtual machine reads and writes the disk in the related technology, a large number of lock requests cause low lock request efficiency and low disk write efficiency are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a schematic diagram of a virtualization cluster in the related art;
FIG. 2 is a diagram illustrating a cluster file system with more locks associated with files in the prior art;
FIG. 3 is a diagram illustrating a file locking process in a cluster file system according to the related art;
FIG. 4 is a diagram illustrating a cluster file system architecture in the related art;
fig. 5 is a block diagram of a hardware structure of a mobile terminal of a disk access method according to an embodiment of the present invention;
FIG. 6 is a flow diagram of a disk access method according to an embodiment of the invention;
FIG. 7 is a schematic diagram of a cluster file system large/small file layout according to an embodiment of the present invention;
FIG. 8 is a bitmap for lock message communication newly added in a file system according to an embodiment of the present invention;
FIG. 9 is a lock release flow diagram according to an embodiment of the present invention;
FIG. 10 is a lock request flow diagram according to an embodiment of the invention;
fig. 11 is a block diagram of a configuration of a disk access apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
To facilitate an understanding of the present invention, a brief description of the related art follows. The following three types of file systems are available for virtual services in the related art:
metadata service node model: one or more (backup redundancy) metadata service nodes exist in the whole cluster, and when other nodes need to modify metadata of the file system, requests are sent to the metadata service nodes for processing. And if the metadata service node is offline or abnormal, starting a standby metadata server and recovering the log, and isolating the original metadata server through an additional nonce link. Under the model, the throughput of the metadata server is required to be higher; when all metadata service nodes are offline, the file system can not be normally used any more; an additional nonce link is also required.
Distributed model: each node can only access the storage node belonging to the node physically, and when the storage on other nodes needs to be accessed, the request is sent to other nodes to be completed. In order to prevent the storage data on a node from being accessed after the node goes offline, the data is usually backed up to a different storage node. Under the model, more nodes are needed to ensure that the whole distributed file system has good performance and stability; additional redundant storage is also required to ensure that data is always accessible at all times. And treatment for brain cracks.
General clustering model: each node can normally access all the storage, and the protection of the metadata is mutually exclusive by means of locks. The lock state is dispersed over one or more nodes in the cluster. When a node needs to access a lock, the node where the lock is located is queried first, and then a lock request is sent to the node. This model is similar to the distributed model except that only locks employ the distributed model, thereby reducing the amount of redundant data, but also introduces some of the disadvantages of the distributed model, such as the need for additional treatment of split brain.
It can be seen from the above description that the above three models are not suitable for medium and small scale virtual service scenarios. The invention mainly provides a new cluster file system model aiming at the scene, and comprehensively considers the stability, the scale, the complexity and the cost.
Virtual service operation model: the files required in the running process of the virtual machine are as follows: configuration files, disk mapping files, snapshot files, wherein configuration files are small files, and disk mapping files and snapshot files are larger files. A typical file system needs to balance between large files and small files when selecting the minimum allocation quota. The allocation frequency is reduced by allocating a larger amount at one time, but when a large number of small files exist in the file system, the allocation formula wastes more storage space. And the allocation of a smaller amount at a time can effectively avoid the waste of space, but also improves the allocation frequency, and the allocation usually has a larger influence on the performance of the file system. In addition, a large amount of fragments are generated after the file is deleted and redistributed, and the use efficiency of the file system is further influenced.
When the virtual machine uses the file as a disk image, the following characteristics are provided:
when a plurality of virtual machines access a file together, a shared read mode of the file is usually requested, and a write operation is not initiated to the file at the moment (a parent file of a multi-leaf cannot be modified after solidification).
When a single virtual machine accesses its non-parent disk file, it typically requests an exclusive mode for that file. (shared disk usage models are now rare) however, a virtual machine will only start on one host at a time, and only have one instance.
The disk image of the virtual machine is usually a large file (thin provisioning) of several hundred M or even G, and a small file is not used as the disk image.
In some special scenarios, a disk image is frequently created and deleted (batch new virtual machines, batch disk backups, etc.). For such a scenario, the resource management method described in the embodiments may be employed to reduce the conflict.
The usage scenario of the virtual machine requires that the host cannot be restarted due to storage exception as much as possible (in the above usage scenario, in order to ensure consistency of metadata, lancet usually chooses to directly restart the host). Restarting the host will cause all virtual machines on that host to shut down abnormally.
In addition, the host machines of medium-sized and small enterprises, institutions and schools are not large in size (generally between 16 and 32 nodes).
Fig. 4 is a schematic diagram of a cluster file system architecture in the related art, and as shown in fig. 4, a conventional cluster file system is composed of a file abstraction layer, a resource management layer, and a log recording layer. The virtual machine disk image is abstracted into a file, and the mapping from the file to a storage logical address is realized by a file image abstraction layer through a resource management layer. And finally, the log recording layer records the metadata to a reserved log area to ensure the consistency of the file system when the metadata is modified by the file image extraction layer and the resource management layer. In addition, for the three functional layers constituting the file system, the three functional layers all need to mutually exclusive access to the key data stored on the shared storage by the file system, so that the protection function provided by the lock module is used.
The invention also greatly optimizes the stability and fault tolerance of the lock module which is very dependent on the cluster file system, and realizes a more efficient resource allocation and use method in the resource management layer aiming at the use scene of the virtual machine. And finally, an accelerated access function which is strongly related to the file system is added to the lock module, so that the performance of the cluster file system is further optimized.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking the example of being operated on a mobile terminal, fig. 5 is a hardware structure block diagram of the mobile terminal of a disk access method according to an embodiment of the present invention. As shown in fig. 5, the mobile terminal 50 may include one or more processors 505 (only one of which is shown in the figure) (the processor 505 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 504 for storing data, and a transmission device 506 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 5 is only an illustration and is not intended to limit the structure of the electronic device. For example, mobile terminal 50 may also include more or fewer components than shown in FIG. 5, or have a different configuration than shown in FIG. 5.
The memory 504 can be used for storing software programs and modules of application software, such as program instructions/modules corresponding to the disk access method in the embodiment of the present invention, and the processor 505 executes various functional applications and data processing by running the software programs and modules stored in the memory 504, so as to implement the above method. The memory 504 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 504 can further include memory located remotely from the processor 505, which can be connected to the mobile terminal 50 over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 506 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal 50. In one example, the transmission device 506 includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 506 can be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a disk access method operating in the mobile terminal is provided, and fig. 6 is a flowchart of a disk access method according to an embodiment of the present invention, as shown in fig. 6, the flowchart includes the following steps:
step S602, after the virtual machine accesses the predetermined disk, controlling the virtual machine to release a lock corresponding to the predetermined disk;
step S604, under the condition of receiving the access request of the virtual machine to the predetermined disk, acquiring a lock in the local memory;
step S606, the virtual machine is controlled to access a predetermined disk.
Through the steps, after the virtual machine accesses the predetermined disk, the lock is only released in the memory of the host, and the disk can not be truly landed, so that when the subsequent virtual machine reads and writes the disk, the lock is only detected in the memory of the host, the virtual machine can access the predetermined disk, the SCSI request is not needed, and the request frequency of the lock is reduced. Therefore, the problems that when the virtual machine reads and writes the disk in the related technology, a large number of lock requests cause low lock request efficiency and low disk write efficiency are solved.
Optionally, before controlling the virtual machine to access the predetermined disk, the method further includes: and establishing a data mapping with a preset size in a preset file system for the metadata and the data of the file corresponding to the preset disk. For example, when expanding data for a file, an 8 megabyte data map is created in the file system for the file at a time, but the size of the file is not modified. In this way, the number of lock requests during file expansion and the possibility of allocating the same resource group to multiple files are further reduced.
For the convenience of understanding the above embodiments, the following detailed description is made:
fig. 7 is a schematic diagram of a layout of large/small files of a cluster file system according to an embodiment of the present invention, and as shown in fig. 7, a relationship between the number of resource group files and occurrence of resource conflicts (lock contention generation) is shown. When there are multiple files in a resource group, then, eventually they can only modify the resource group metadata serially when all files are simultaneously expanded. To reduce such unnecessary conflicts, resource allocation can be optimized mainly from three aspects:
when creating the file, the resource group cached in the local is preferentially considered to be used, and meanwhile, whether the file metadata is not distributed in the resource group is checked. When both conditions are satisfied, file metadata may be selected for allocation on the resource group. Otherwise, the local cache limit is relaxed, simple prejudgment is carried out on resource groups which are not cached locally (the data may not be latest because of no locking), if the locking request times and the time consumption of the resource groups on the host are lower than the average value, the latest data is checked after the locking is requested, and the conflict condition and the file metadata distribution condition are accurately judged again. And if the condition is still not met, notifying the upper layer that the file is failed to be created.
When the file is allocated, the data is basically consistent with the above process, and whether the file metadata is already allocated on the resource group or not is not required to be considered. In addition, if all the resource groups are checked, the resource group with the minimum conflict in the record can not be selected to allocate data to the file.
Finally, when expanding data for a file, an 8 megabyte data map is created in the file system for the file at a time, but the size of the file is not modified. In this way, the lock request times during file expansion and the possibility of distributing the same resource group to a plurality of files are further reduced.
Optionally, after controlling the virtual machine to release the lock corresponding to the predetermined disk, the method further includes: in the event that it is determined that the lock has been cached in the local memory for more than a predetermined amount of time, the host releases the lock and updates an identifier indicating whether the lock was released by the host. Optionally, before updating the identifier indicating whether the lock is released by the host, the method further includes: the identification is stored using predetermined bits on a predetermined disk.
Optionally, in a case that an access request of the virtual machine to the predetermined disk is received, acquiring the lock in the local memory includes: and acquiring the lock in the local memory under the condition of receiving the access request of the virtual machine to the preset disk within preset time.
For the convenience of understanding the above embodiments, the following detailed description is made:
when writing operation is performed in the related art, other hosts hardly request to read and write the file. Therefore, the traditional lock using mode is selected to be modified, when the lock is released, the lock is only released in the memory of the host, and the disk is not really dropped. This means that a notification mechanism is also needed to drop the lock cache on demand and in time, and a notification mechanism for releasing the lock via the disk can be selected.
Fig. 8 is a bitmap for lock message communication newly added in a file system according to an embodiment of the present invention, and as shown in fig. 8, a layout of a disk notification bitmap (hereinafter referred to as a bitmap) is described, a 4k region is divided on a disk, one bit corresponds to one lock notification bit, and notification of 32768 locks can be provided in total. The number of the cluster file system locks is mainly determined by the number of the disk files of the virtual machine, and assuming that one disk uses a space of 50G, thirty-thousand locks can support management of a storage space of more than 1000T and are enough for medium-sized and small-sized virtual scenes.
FIG. 9 is a lock release flow diagram according to an embodiment of the invention, as shown in FIG. 9, describing the flow of how a host senses and releases a lock.
1) The host reads the content of the bitmap once every second, obtains all set lock ID sets in real time and caches the lock ID sets in the memory.
2) And traversing and comparing the set lock with the lock held by the lock, and recording the lock if the lock is not held by the lock (the lock in the memory is released but the lock is not dropped) and the lock holding time exceeds the minimum lock holding time (the lock is not jumped between the hosts for protection).
3) And requesting SCSI protection, releasing the lock which is not held any more (disk dropping), and releasing the SCSI protection after updating the flag bit corresponding to the bitmap.
FIG. 10 is a lock request flow diagram according to an embodiment of the invention, as shown in FIG. 10, describing the flow of a host requesting a lock.
1) It is checked whether the lock is already cached in memory.
a) If there is a cached lock and the owner of the lock is self, then authorization is again directed.
b) And if the cached lock does not exist, reading the locked state of the disk.
i. The lock is occupied by other hosts, and the mark in the bitmap is not set, and the bitmap is recorded and is modified after a period of time, the SCSI lock is requested to be protected in a unified mode.
And ii, periodically reading the state of the lock on the disk, requesting SCSI protection to acquire the lock after finding that the lock is released, and caching the lock after successful acquisition.
An abnormal flow: when the lock holding node is abnormal, the flag bit of the lock notification bitmap cannot be cleared. The original holder (abnormal node) of the lock is cleared up until the recovery flow of the cluster file system is triggered, and the node holding the lock again changes the lock releasing action into direct disk dropping (only once) because the flag bit is not cleared. This loss of efficiency is acceptable without introducing additional recovery procedures.
For the cluster file system abstraction layer, it is not desirable to look directly at the lock entity, and it should be abstracted to access to a file or a group of resources, so there may be behaviors that encapsulate:
1) When a file is requested to be accessed, the bottom layer also prepares the inode and the latest data of the indirect blocks (read from the storage) in the memory for the host while granting the file access authority.
2) When a resource group is requested to be accessed, the bottom layer also prepares the latest data (read from the storage) of the resource group in the memory for the host while granting the access right to the resource group.
3) After the access is finished, the bottom layer releases the access right and simultaneously synchronizes the modification to the storage if the content of the file and the resource is modified.
4) And when the file and the resource group are released, destroying the data of the resource in the memory.
Through the encapsulation, the host computer is ensured to obtain the required data after requesting the access authority of the file and the resource.
The improvement forms the main content of the invention, greatly simplifies the use complexity of the cluster file system, weakens the limitation and improves the efficiency under the medium and small-scale virtualization service use scene.
Through the embodiment, the invention relates to improvement of a cluster file system which does not depend on network communication and is suitable for a virtual machine. The cluster file system is required to be established on a magnetic array or shared storage supporting SCSI protocol, no additional communication link is needed, and the virtual machine can efficiently and stably use the disk mapping file. The method can improve the traditional cluster file system through a data layout and mutual exclusion access mode specially designed for the use scene of the virtual machine, strip the dependence on the network, reduce the complexity of the whole system and improve the access efficiency and stability as much as possible. The method is suitable for the use scene of the file system cluster with small or moderate scale.
In particular, the present invention is directed to a general cluster file system, which is designed mainly for a general usage scenario. When the virtual machine uses the file systems, all functions of the file systems (described above) are not involved, and aiming at the characteristics, a general cluster file system is improved, additionally introduced networks, funce links and the like are reduced, a method for resource allocation and data mutual exclusion is improved, and the stability of the cluster file system is further enhanced, and meanwhile, the performance of the virtual machine running service is improved.
The invention adopts the following technical scheme that the invention comprises two parts:
one, a network independent cluster lock module is implemented. The module gives consideration to the characteristics of two aspects of lock request efficiency and stability, and simultaneously improves the efficiency of accessing data in a critical zone by a file abstraction layer and a resource management layer.
And secondly, a resource allocation module suitable for the virtual machine service is realized. The module optimizes the scene of using the storage of the virtual machine, and has good resource allocation efficiency and smaller resource conflict rate.
Compared with the prior art, the method does not depend on a network link, does not need an additional fence link, and only needs a magnetic array or shared storage to support the SCSI protocol. And the method has better stability and efficiency in the scene of running the virtual machine service.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
In this embodiment, a disk access device is further provided, and the device is used to implement the foregoing embodiments and preferred embodiments, and details are not described again after the description is given. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 11 is a block diagram of a configuration of a disk access apparatus according to an embodiment of the present invention, as shown in fig. 11, the apparatus including:
the cache module 112 is configured to, after the virtual machine accesses the predetermined disk, control the virtual machine to release a lock corresponding to the predetermined disk and cache the lock in a local memory of a host where the virtual machine is located;
an obtaining module 114, connected to the control module 112, configured to obtain the lock in the local memory when receiving an access request of the virtual machine to the predetermined disk;
and a control module 116, connected to the obtaining module 114, for controlling the virtual machine to access the predetermined disk.
Optionally, the control module 116 is further configured to, before controlling the virtual machine to access the predetermined disk, establish a data map with a predetermined size in the predetermined file system for the metadata and the data of the file corresponding to the predetermined disk.
Optionally, the caching module 112 is further configured to, after controlling the virtual machine to release the lock, if it is determined that the lock is cached in the local memory for more than a predetermined time period, release the lock by the host and update the identifier indicating whether the lock is released by the host.
Optionally, the cache module 112 is further configured to store the identifier with a predetermined bit on a predetermined disk before updating the identifier indicating whether the lock is released by the host.
Optionally, the obtaining module 114 is further configured to obtain the lock in the local memory when receiving an access request of the virtual machine to a predetermined disk within a predetermined time.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
An embodiment of the present invention further provides a storage medium including a stored program, where the program executes any one of the methods described above.
Alternatively, in the present embodiment, the storage medium may be configured to store program codes for performing the following steps:
s1, after a virtual machine accesses a preset disk, controlling the virtual machine to release a lock corresponding to the preset disk and caching the lock in a local memory of a host where the virtual machine is located;
s2, under the condition that the access request of the virtual machine to the preset disk is received, acquiring the lock in the local memory and caching the lock in the local memory of the host where the virtual machine is located;
and S3, controlling the virtual machine to access the preset disk.
Optionally, the storage medium is further arranged to store program code for performing the steps of: before controlling the virtual machine to access the predetermined disk, the method further comprises:
s1, establishing data mapping with a preset size in a preset file system for metadata and data of a file corresponding to a preset disk.
Optionally, the storage medium is further arranged to store program code for performing the steps of: after controlling the virtual machine to release the lock corresponding to the predetermined disk, the method further includes:
s1, under the condition that the cache time of the lock in the local memory is determined to exceed the preset time, the host releases the lock and updates an identifier for indicating whether the lock is released by the host.
Optionally, the storage medium is further arranged to store program code for performing the steps of: before updating the identifier for indicating whether the lock is released by the host, the method further comprises the following steps:
s1, storing the identification by using a preset bit on a preset magnetic disk.
Optionally, the storage medium is further arranged to store program code for performing the steps of: in a case where the access request of the virtual machine to the predetermined disk is received, acquiring the lock in the local memory includes:
s1, under the condition that the access request of the virtual machine to the preset disk is received within preset time, the lock in the local memory is obtained.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide a processor configured to execute a program, where the program executes to perform any of the steps in the method.
Optionally, in this embodiment, the program is configured to perform the following steps:
s1, after a virtual machine accesses a preset disk, controlling the virtual machine to release a lock corresponding to the preset disk and caching the lock in a local memory of a host where the virtual machine is located;
s2, under the condition that the access request of the virtual machine to the preset disk is received, acquiring the lock in the local memory and caching the lock in the local memory of the host where the virtual machine is located;
and S3, controlling the virtual machine to access the preset disk.
Optionally, in this embodiment, the program is configured to perform the following steps: before controlling the virtual machine to access the predetermined disk, the method further comprises the following steps:
s1, establishing data mapping with a preset size in a preset file system for metadata and data of a file corresponding to a preset disk.
Optionally, in this embodiment, the program is configured to perform the following steps: after controlling the virtual machine to release the lock corresponding to the predetermined disk, the method further includes:
s1, under the condition that the cache time of the lock in the local memory is determined to exceed the preset time, the host releases the lock and updates an identifier for indicating whether the lock is released by the host.
Optionally, in this embodiment, the program is configured to perform the following steps: before updating the identifier for indicating whether the lock is released by the host, the method further comprises the following steps:
s1, storing the identification by using a preset bit on a preset magnetic disk.
Optionally, in this embodiment, the program is configured to perform the following steps: in a case where the access request of the virtual machine to the predetermined disk is received, acquiring the lock in the local memory includes:
s1, under the condition that the access request of the virtual machine to the preset disk is received within preset time, the lock in the local memory is obtained.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. A disk access method, comprising:
after a virtual machine accesses a preset disk, controlling the virtual machine to release a lock corresponding to the preset disk, and caching the lock in a local memory of a host where the virtual machine is located, wherein the lock comprises at least one of an inode lock, a resource group lock and a transaction lock;
under the condition of receiving the access request of the virtual machine to the preset disk, acquiring the lock in the local memory;
controlling the virtual machine to access the predetermined disk;
controlling the virtual machine to release a lock corresponding to the predetermined disk, and caching the lock in a local memory of a host where the virtual machine is located includes:
reading the content of the bitmap once at intervals of preset duration to obtain all set lock ID sets, and caching the set lock ID sets in the local memory;
and comparing the set lock ID set with a lock ID set held by the virtual machine, determining a lock ID which is not held by the virtual machine and the lock holding time of which exceeds the minimum lock holding time, controlling the virtual machine to release a lock corresponding to the determined lock ID, and caching the released lock in a local memory of a host where the virtual machine is located.
2. The method of claim 1, prior to controlling the virtual machine to access the predetermined disk, further comprising:
and establishing data mapping with a preset size in a preset file system for the metadata and the data of the file corresponding to the preset disk.
3. The method according to claim 1, further comprising, after controlling the virtual machine to release the lock corresponding to the predetermined disk:
and under the condition that the time that the lock is cached in the local internal memory is determined to exceed the preset time, the host releases the lock and updates the identifier for indicating whether the lock is released by the host.
4. The method of claim 3, further comprising, prior to updating the identification indicating whether the lock was released by the host:
storing the identification using predetermined bits on the predetermined disk.
5. The method of claim 1, wherein acquiring the lock in the local memory upon receiving an access request from the virtual machine to the predetermined disk comprises:
and acquiring the lock in the local memory under the condition of receiving the access request of the virtual machine to the preset disk within preset time.
6. A disk access device, comprising:
the cache module is used for controlling the virtual machine to release a lock corresponding to a predetermined disk after the virtual machine accesses the predetermined disk, and caching the lock in a local memory of a host where the virtual machine is located, wherein the lock comprises at least one of an inode lock, a resource group lock and a transaction lock;
an obtaining module, configured to obtain the lock in the local memory when receiving an access request of the virtual machine to the predetermined disk;
the control module is used for controlling the virtual machine to access the preset disk;
controlling the virtual machine to release a lock corresponding to the predetermined disk, and caching the lock in a local memory of a host where the virtual machine is located, including:
reading the content of the bitmap once at intervals of preset duration to obtain all set lock ID sets, and caching the set lock ID sets in the local memory;
and comparing the set lock ID set with a lock ID set held by the virtual machine, determining the lock ID which is not held by the virtual machine any more and the lock holding time of which exceeds the minimum lock holding time, controlling the virtual machine to release the lock corresponding to the determined lock ID, and caching the released lock in a local memory of a host where the virtual machine is located.
7. The apparatus of claim 6, wherein the control module is further configured to establish a data map of a predetermined size in a predetermined file system for metadata and data of a file corresponding to the predetermined disk before controlling the virtual machine to access the predetermined disk.
8. The apparatus of claim 6, wherein the caching module is further configured to, after controlling the virtual machine to release the lock, if it is determined that the lock is cached in the local memory for more than a predetermined time period, the host releases the lock and updates an identifier indicating whether the lock is released by the host.
9. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 5.
10. A mobile terminal, characterized in that the mobile terminal comprises one or more processors for running a program, wherein the program when running performs the method of any of claims 1 to 5.
CN201710875562.4A 2017-09-25 2017-09-25 Disk access method and device Active CN109558205B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710875562.4A CN109558205B (en) 2017-09-25 2017-09-25 Disk access method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710875562.4A CN109558205B (en) 2017-09-25 2017-09-25 Disk access method and device

Publications (2)

Publication Number Publication Date
CN109558205A CN109558205A (en) 2019-04-02
CN109558205B true CN109558205B (en) 2022-11-29

Family

ID=65862553

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710875562.4A Active CN109558205B (en) 2017-09-25 2017-09-25 Disk access method and device

Country Status (1)

Country Link
CN (1) CN109558205B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112003946B (en) * 2020-08-26 2023-05-12 新华智云科技有限公司 Data disc-falling method and data disc-falling system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609419A (en) * 2009-06-29 2009-12-23 北京航空航天大学 The data back up method and the device of the migration of virtual machine consistently online
CN102063338A (en) * 2010-12-28 2011-05-18 华为技术有限公司 Method and device for requesting exclusive resource
CN106406981A (en) * 2016-09-18 2017-02-15 深圳市深信服电子科技有限公司 Disk data reading/writing method and virtual machine monitor
CN106648909A (en) * 2016-10-13 2017-05-10 华为技术有限公司 Management method and device for dish lock and system
CN107045530A (en) * 2017-01-20 2017-08-15 华中科技大学 A kind of method that object storage system is embodied as to local file system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609419A (en) * 2009-06-29 2009-12-23 北京航空航天大学 The data back up method and the device of the migration of virtual machine consistently online
CN102063338A (en) * 2010-12-28 2011-05-18 华为技术有限公司 Method and device for requesting exclusive resource
CN106406981A (en) * 2016-09-18 2017-02-15 深圳市深信服电子科技有限公司 Disk data reading/writing method and virtual machine monitor
CN106648909A (en) * 2016-10-13 2017-05-10 华为技术有限公司 Management method and device for dish lock and system
CN107045530A (en) * 2017-01-20 2017-08-15 华中科技大学 A kind of method that object storage system is embodied as to local file system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
虚拟机无法打开磁盘 "或者某一个快照所依赖的磁盘 原因:未能锁定文件"的解决办法;huchunlinnk;《https://blog.csdn.net/huchunlinnk/article/details/38365527》;20140804;第1-2页 *

Also Published As

Publication number Publication date
CN109558205A (en) 2019-04-02

Similar Documents

Publication Publication Date Title
US10817386B2 (en) Virtual machine recovery method and virtual machine management device
US20210374157A1 (en) System and method for near-synchronous replication for object store
US8775755B2 (en) Peer-to-peer transcendent memory
KR101833114B1 (en) Fast crash recovery for distributed database systems
US9424140B1 (en) Providing data volume recovery access in a distributed data store to multiple recovery agents
US9697226B1 (en) Network system to distribute chunks across multiple physical nodes
CN111651305B (en) Virtual machine backup method, virtual machine backup assembly and cloud computing system
US20190199794A1 (en) Efficient replication of changes to a byte-addressable persistent memory over a network
CN113568566A (en) Method, host device and storage server for seamless migration of simple storage service by using index object
CN115277145B (en) Distributed storage access authorization management method, system, device and readable medium
CN108762982B (en) A kind of database restoring method, apparatus and system
CN114185558A (en) Native application master selection method and device based on K8s and storage medium
US11080909B2 (en) Image layer processing method and computing device
CN108271420B (en) Method for managing files, file system and server system
CN110941591A (en) File deletion method, device and equipment and readable storage medium
CN111399753A (en) Method and device for writing pictures
CN109558205B (en) Disk access method and device
CN117170820A (en) Configuration sharing method, system, terminal and storage medium of cluster node
US10848405B2 (en) Reporting progress of operation executing on unreachable host
CN105491101B (en) The treating method and apparatus of data
CN113194121B (en) File backup method between electronic devices
CN111400302B (en) Modification method, device and system for continuous storage data
US20210248108A1 (en) Asynchronous data synchronization and reconciliation
US20240036989A1 (en) Storage system and management method
CN116301624A (en) Storage area creation method, system, electronic device and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant