WO2016070641A1

WO2016070641A1 - Data storage method and device, and data reading method and device

Info

Publication number: WO2016070641A1
Application number: PCT/CN2015/083472
Authority: WO
Inventors: 胡振; 崔丕锁; 于维纳; 陆剑峰; 李世亮; 秦延涛
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-11-04
Filing date: 2015-07-07
Publication date: 2016-05-12
Also published as: CN105630401A

Abstract

A data storage method and device, and data reading method and device, the storage method comprising: a server receives a write request for data (S102); upon being triggered by the write request, the server writes the data to a sector of a local virtual hard disk (VHD) file and stores the data (S104). The method and device solve the problems of RAW format occupying a large amount of disk space, inconvenient backup, and not supporting a sub-mirror, thus enabling allocation of disk space as needed, convenient backup, and supporting a snapshot and sub-mirror.

Description

Data storage method, device, and reading method, device

Technical field

The present invention relates to the field of communications, and in particular, to a data storage method, device, and reading method and device

Background technique

The network diskless system is a technology that saves the operating system image on the server. The client does not have a hard disk locally or does not use a local hard disk. The preboot execution environment (PXE) is used to boot from the network. The client operating system is stored on the server and requires an efficient and flexible management method. The traditional method is to use a virtual hard disk in the raw RAW format, and then use the Internet Small Computer System Interface (iSCSI). Sharing to the client, using RAW format takes up a lot of disk space, inconvenient to backup, etc. For example, using RAW mainly has the following shortcomings: (1) wasting storage space: using the bare format RAW format hard disk needs to allocate all the space in advance If you create a 10G hard disk for the user, you must first create a 10G file on the server. You cannot allocate space as needed by the user. (2) Inconvenient maintenance: The virtual hard disk file is generally larger. And the RAW format does not support snapshots, so backups can only be copied, The way of pasting is slow and wastes space, and maintenance is inconvenient.

In the related art, the RAW format has a problem that the disk space is large, the backup is inconvenient, and the sub-image is not supported, and an effective solution has not been proposed.

Summary of the invention

Embodiments of the present invention provide a method, an apparatus, and a reading method and apparatus for saving data to solve at least one of the above problems.

According to an aspect of the embodiments of the present invention, a data saving method is provided, including: a server receives a write request for data; and under the trigger of the write request, the server writes the data to a local virtual The sector in the virtual hard disk (VHD) file is saved and saved.

Further, before the server writes the operating system data into a sector in the local VHD file, the method includes: the server determines whether a sector corresponding to the write request has been allocated, where In the case of being allocated, the server allocates the sector for the data; in the event that the determination has been assigned, the server is triggered to write the data to the sector.

Further, the server determines whether a sector corresponding to the write request has been allocated, and the server obtains a Block allocation table (BAT) table from the local VHD file; the server A determination is made as to whether a sector corresponding to the write request has been allocated according to the BAT table.

Further, the method further includes: sending, when the operating system image file changes, an incremental portion of the operating system image file that is changed to the client.

Further, the method further includes: the server receiving a read request for the VHD file; reading data corresponding to the read request from a difference disk; and confirming that the data corresponding to the read request is in the difference In the case where the disk is not recorded, the data is read from the parent disk of the difference disk, wherein the data stored in the difference disk is data changed in the parent disk.

Further, the method further includes: mounting the VHD file as a raw device file, and providing the operating system image file to the client.

According to another aspect of the embodiments of the present invention, a method for reading data includes: receiving, by the server, a read request for an operating system image file; and triggering, by the read request, the server The data corresponding to the read request is read in the local virtual hard disk VHD file.

Further, the reading, by the server, the data corresponding to the read request from the local VHD file includes: reading data corresponding to the read request from the difference disk; and confirming that the data corresponding to the read request is in the difference disk If the data is not recorded, the data is read from the parent disk of the difference disk, wherein the data stored in the difference disk is data changed in the parent disk.

According to an aspect of an embodiment of the present invention, there is provided a data saving apparatus, the apparatus being applied to a server, comprising: a receiving module configured to receive a write request for data; and a writing module configured to be in the write Under the trigger of the request, the data is written to the sector in the local virtual hard disk VHD file and saved.

According to another aspect of the embodiments of the present invention, there is also provided a data reading device, the device being applied to a server, comprising: a receiving module configured to receive a read request for an operating system image file; and a reading module And being configured to read data corresponding to the read request from a local VHD file under the trigger of the read request.

Through the embodiment of the present invention, the server receives the write request for the data; under the trigger of the write request, the server writes the data to the sector in the local virtual hard disk VHD file and saves it. The RAW format in the related art solves the problem that the RAW format occupies a large disk space, is inconvenient to backup, and does not support the sub-image. The disk space is allocated on demand, the backup is convenient, and the snapshot and sub-image are supported.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart of a method for saving data according to an embodiment of the present invention;

2 is a flow chart of a method of reading data according to an embodiment of the present invention;

3 is a structural block diagram of a data saving apparatus according to an embodiment of the present invention;

4 is a block diagram showing the structure of a data reading apparatus according to an embodiment of the present invention;

FIG. 5 is a structural diagram of a VHD file according to an embodiment of the present invention; FIG.

6 is a flow chart of a VHD read in accordance with an embodiment of the present invention;

7 is a flow chart of writing a VHD file according to an embodiment of the present invention;

FIG. 8 is a flow chart of creating a sub-image of a VHD according to an embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

In this embodiment, a method for saving an operating system image file is provided. FIG. 1 is a flowchart of a method for saving data according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:

Step S102, the server receives a write request for data;

Step S104, under the trigger of the write request, the server writes the data into the sector in the local virtual hard disk VHD file and saves it.

Through the above steps, the server uses the VHD format to write and manage the operating system image file. Compared with the related art, when the RAW format is used, the storage space is wasted, and the maintenance is inconvenient, and the above steps solve the above problems, thereby achieving the above problem. Allocate disk space on demand, easy backup, support for snapshots and submirrors.

In an optional embodiment, before the server writes the operating system data into the sector in the local VHD file, the server determines whether the sector corresponding to the write request has been allocated, wherein, in the case where the determination is not allocated, The server allocates sectors for the data; in the case where it is determined that it has been allocated, the trigger server writes operating system data to the sector. Therefore, the server allocates sectors according to the write request, realizing on-demand allocation and dynamically allocating disk space.

In the above steps, the server needs to determine whether the sector corresponding to the write request has been allocated. In an optional embodiment, the server obtains the BAT table from the local VHD file, and the server determines the sector corresponding to the write request according to the BAT table. Has it been assigned?

In an alternative embodiment, the incremental portion of the changes in the operating system image file is sent to the client when the operating system image file changes. Through this step, only the incremental part of the operating system image file is sent to the client, which improves storage efficiency and reduces storage space.

When the server receives the read request for the VHD file, in an optional embodiment, the data corresponding to the read request is first read from the difference disk, and if the data corresponding to the read request is confirmed to be unrecorded in the difference disk, The data is read from the parent disk of the differencing disk, where the data stored in the differencing disk is the data that changed in the parent disk. This saves storage space.

In an alternative embodiment, the VHD file is mounted as a raw device file and an operating system image file is provided to the client. The raw device file is equivalent to a physical hard disk.

In the embodiment, a data reading method is also provided. FIG. 2 is a flowchart of a data reading method according to an embodiment of the present invention. As shown in FIG. 2, the process includes the following steps:

Step S202, the server receives a read request for an operating system image file.

Step S204, the server reads the data corresponding to the read request from the local virtual hard disk VHD file under the trigger of the read request.

Through the above steps, the server saves the operating system image in the VHD format. Compared with the related art, when the RAW format is adopted, the storage space is wasted, and the maintenance is inconvenient, the above steps solve the above problem, thereby achieving the disk space allocated on demand. , the effect of using snapshots for backups.

When the server reads data from the local VHD file, in an optional embodiment, the data corresponding to the read request is first read from the difference disk, and if the data corresponding to the read request is confirmed to be unrecorded in the difference disk, The data is read from the parent disk of the differencing disk, where the data stored in the differencing disk is the data that changed in the parent disk. Thereby improving the efficiency of reading files.

3 is a structural block diagram of a data saving apparatus according to an embodiment of the present invention. The apparatus is applied to a server. As shown in FIG. 3, the apparatus includes: a receiving module 32 configured to receive a write request for an operating system image file; The write module 34 is configured to write the operating system data to the sector in the local virtual hard disk VHD file and save it under the trigger of the write request.

4 is a structural block diagram of a data reading apparatus according to an embodiment of the present invention. The apparatus is applied to a server. As shown in FIG. 4, the apparatus includes: a receiving module 42 configured to receive a read request for an operating system image file. The read module 44 is configured to read the operating system data from the local VHD file under the trigger of the read request.

It should be noted that each of the above modules may be implemented by software or hardware. For the latter, the foregoing may be implemented by, but not limited to, the foregoing modules are all located in the same processor; or, the above modules are respectively located. In the first processor, the second processor, and the third processor.

For the above problems in the related art, the following description will be made in conjunction with an alternative embodiment which combines the above-described alternative embodiments and alternative embodiments thereof.

Interpretation of related technical terms:

VHD: Microsoft Virtual Hard Disk format, a virtual disk format that supports dynamic and nutrient disks. The Microsoft Hyper-v virtualization platform uses VHD as a virtual machine hard disk. Similar formats include VMDK and QCOW2.

PXE: A network boot protocol. This is implemented in a general network card.

iPXE: An upgraded version of the PXE protocol.

Networkless disk: Boot the computer from the network through PXE or iPXE. All data is saved on the remote server. There is no hard disk in the local.

iSCSI: A storage protocol that shares block devices (such as disks) over an IP network. The client uses the iSCSI shared hard disk to be the same as the local hard disk. It can partition and format the disk.

This alternative embodiment develops a method for saving and managing images using a virtual hard disk (VHD) format. The VHD format has the advantages of space-on-demand allocation, snapshot creation and encryption, and the like. The image is migrated directly to the virtualization platform.

For the disadvantage of saving the image in the RAW format, the optional embodiment is designed to save the image file in the VHD format. In order to use the VHD format file, the optional embodiment has a method for directly reading and writing a VHD file without using a third-party tool; After receiving the VHD read and write request, first open the VHD file to obtain the BAT of the VHD. The table determines whether the corresponding sector is allocated in the VHD file and the position of the sector in the file according to the BAT table. If the sector corresponding to the write request is not allocated in the VHD file, the corresponding data block is to be allocated and modified. BAT table, and then write the data to the corresponding location of the file; if you want to read a block x, the system will first read the bat entry of the difference disk x block. If the value is 0xFFFFFFFF, indicating that the data is not recorded in the difference disk, then the xth block of the parent disk of the difference disk should be read; if the value of the bat entry is not 0xFFFFFFFF, whether the x block in the batmap can be used is full. (or analyze the bitmap in the block data area) to determine whether it needs to penetrate into the parent disk to make up the difference. If it is full, it does not need to process the parent disk; if it is not full, then look at the bitmap in the data area of the block.

Bit

0, 0 is penetrated into the parent disk to make up the difference, and 1 is directly read.

A major feature of VHD files is that disk space is allocated on demand. When a VHD file is created, no data block (disk space) is allocated. When the VHD receives a write disk operation, it will detect the corresponding bat table. If the value of the bat table is 0xFFFFFFFF , indicating that the corresponding data block is not allocated in the VHD file, then allocate the data block by increasing the size of the VHD file (by writing a number of zeros at the end of the file), and modifying the corresponding value in the bat table. The offset of the new data block in the VHD file;

Another feature of VHD files is that they support the snapshot function. The snapshot technology can be used to quickly back up data. It is much faster than copying and pasting files directly. VHD snapshots are created by creating sub-images of VHDs (also called differential disks). (Because the VHD sub-image is created without assigning data blocks, only a small content such as a VHD file header and a BAT table is created, so the speed is fast). After the sub-image is created, the read and write operations are performed only for the sub-image, and the parent image is no longer operated. The results of reading and writing are stored in the sub-image, which plays a role in the backup of the parent image;

In order to enable the VHD to provide a diskless boot service through the iSCSI, the optional embodiment introduces a zhong xing virtualization environments (ZXVE) platform (operating system, ZTE virtualization platform), and the VHD can be hung by the ZXVE. Loaded into a raw device file (equivalent to a physical hard disk), after being turned into a bare device, it can provide external services through iSCSI;

The steps to use a VHD are as follows:

1. The server mounts the VHD as a bare device;

2. The VHD raw device is shared by iSCSI;

3. The client starts up with the PXE network, obtains the address from the DHCP server for iPXE startup, and enters the iPXE to select the operating system to be started to start by the shared iSCSI;

4. The client sends a read and write request for the iSCSI disk to the server.

5. After receiving the request, the server reads and writes the VHD and returns the result to the client (for detailed reading and writing of the VHD, please refer to the attached figure).

The network diskless system needs to occupy a large amount of network bandwidth, which has high requirements on the network and affects the user experience. This alternative embodiment designs a local data cache and incremental update method to improve access to the network and improve diskless users. Use experience

When the client needs to fetch data from the server, the system checks whether the data requested by the user exists in the buffer. If the data is cached, the data is directly retrieved from the buffer area and returned to the user, and the data does not need to be fetched through the network;

If there is no data required by the user in the buffer, the server in the network requests the data, and saves the data in the buffer after the data is acquired, so that it can be used again until all the data is cached locally. Great savings in network bandwidth.

After all the data is cached, there may be cases where the cached data may be inconsistent in the server data (such as the administrator updating the image on the server, installing the software, etc.), and the client requests to update the system after the administrator modifies the server parent image. Calculate the variable sector of the sub-image intersecting parent image, and send the variable part to the client. The client writes the changed part to the local buffer according to the server-side data, and restarts the computer after the client change part is updated.

In order to make the objects, technical solutions and advantages of the present embodiments more clear, the present exemplary embodiments will be further described in detail below with reference to the accompanying drawings.

FIG. 5 is a structural diagram of a VHD file according to an embodiment of the present invention. As shown in FIG. 5, the structure of a VHD file needs to be parsed for directly reading and writing a VHD file;

FIG. 6 is a flow chart of VHD reading according to an embodiment of the present invention. As shown in FIG. 6, the process includes the following steps:

Step S602, opening the VHD;

Step S604, acquiring a BAT table;

Step S606, determining the content of the corresponding index in the BAT table according to the sector number to be read;

Step S608, it is determined whether the value in the BAT table is 0xFFFFFFFF, if the determination is yes, step S610 is performed, if the determination is no, step S614 is performed;

Step S610, indicating that the content of the sector to be read is 0;

Step S612, returning the sector content to 0 to the user;

Step S614, reading the corresponding position of the VHD according to the content of the BAT table, the read size is 2M, and the first sector represents the use bitmap of each sector in the data block;

Step S616, it is determined whether the sector bitmap value is 1, if the determination is yes, step S618 is performed, if the determination result is no, step S620 is performed;

Step S618, returning the content of the read sector to the user;

Step S620, finding a parent node of the VHD file, and reading the parent node according to the reading process to obtain the sector content;

Step S622, return.

Further, to read a certain data block x, the system first reads the bat entry of the difference disk x block. If the value is 0xFFFFFFFF, indicating that the data is not recorded in the difference disk, then the xth block of the parent disk of the difference disk should be read; if the value of the bat entry is not 0xFFFFFFFF, whether the x block in the batmap can be used is full. (or analyze the bitmap in the block data area) to determine whether it needs to penetrate into the parent disk to make up the difference. If it is full, it does not need to process the parent disk; if it is not full, then look at the bitmap in the data area of the block.

Bit

FIG. 7 is a flowchart of writing a VHD file according to an embodiment of the present invention. As shown in FIG. 7, the process includes the following steps:

Step S702, opening the VHD;

Step S704, acquiring a BAT table;

Step S706, determining the content of the corresponding index in the BAT table according to the sector number to be read;

Step S708, it is determined whether the value in the BAT table is 0xFFFFFFFF, if the determination is yes, step S710 is performed, if the determination is no, step S714 is performed;

Step S710, indicating that the storage space of the data block where the sector to be written is not allocated, and increasing the size of the VHD file to allocate space for the data block;

Step S712, after the data block is allocated, the BAT table of the VHD file and the batmap of the data block are modified, and the user data is written into the corresponding position of the file;

Step S714, modifying the batmap of the data block, and writing the user data to the location corresponding to the file.

Further, when writing a certain sector, it is first determined whether the data block in which the sector is located has been allocated. If it is allocated, the data is directly written to the corresponding location and the batmap of the data block is modified, if the data block is not allocated. Then increase the size of the VHD file to allocate the data block to modify the BAT table and then write the data;

FIG. 8 is a flowchart of creating a sub-mirror of a VHD according to an embodiment of the present invention, filling a sub-mirror VHD header according to attributes such as a capacity of a mirror, a block size, and the like, and writing a UUID and path information of the parent image to the sub-image so as to be able to pass the sub-mirror. Find the parent image.

In summary, the RAW format is replaced by a dynamic virtual disk VHD according to an optional embodiment of the present invention, which can effectively improve the utilization and maintainability of the storage space, and the system design a local cache system to reduce the occupation of the network by the client. After enabling the local cache, in addition to the first request data, you need to download data directly from the network, and even disconnect the network after the data is fully cached, reducing the network burden. The new VHD method can effectively solve the problem of using the RAW format, greatly improving the storage space utilization and maintenance; at the same time, using the network without disk has higher requirements on the network, especially in the case of multiple clients, the network response is slow. Involving the customer experience, the method provided by the optional embodiment can cache the server data and the client write back to a local hard disk (if any) to reduce the network pressure, thereby improving the user experience. With the convenience of diskless convenience; at the same time, when the server-side operating system is updated, the client can update the changed part of the image to the local by incremental update method, which greatly improves the update speed.

In another embodiment, software is also provided for performing the technical solutions described in the above embodiments and preferred embodiments.

In another embodiment, a storage medium is further provided, wherein the software includes the above-mentioned software, including but not limited to: an optical disk, a floppy disk, a hard disk, an erasable memory, and the like.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

The foregoing embodiments of the present invention are applied to the field of communications, and solve the problem that the RAW format in the related art has a large disk space, is inconvenient to backup, and does not support sub-mirror, thereby achieving disk space allocation on demand, convenient backup, and support. The effect of snapshots and submirrors.

Claims

A method of saving data, including:

The server receives a write request for data;

Upon the triggering of the write request, the server writes the data to a sector in the local virtual hard disk VHD file and saves it.
The method of claim 1 wherein said server prior to writing said data to a sector in a local VHD file comprises:

Determining, by the server, whether a sector corresponding to the write request has been allocated, wherein, in the case where the determination is not assigned, the server allocates the sector for the data; in the case where it is determined that it has been allocated And triggering the server to write the data to the sector.
The method of claim 2, wherein the server determines whether a sector corresponding to the write request has been allocated comprises:

The server acquires a block allocation table BAT table from the local VHD file;

The server determines whether a sector corresponding to the write request has been allocated according to the BAT table.
The method of claim 3, wherein the method further comprises:

When the operating system image file changes, the incremental portion of the operating system image file is sent to the client.
The method of claim 1 wherein the method further comprises:

The server receives a read request for the VHD file;

Reading data corresponding to the read request from the difference disk;

And in the case that the data corresponding to the read request is not recorded in the difference disk, the data is read from a parent disk of the difference disk, wherein the data stored in the difference disk is the parent The changed data in the disk.
The method of any of claims 1 to 5, wherein the method further comprises:

The VHD file is mounted as a raw device file, and the operating system image file is provided to the client.
A method of reading data, including:

The server receives a read request for an operating system image file;

The server reads the data corresponding to the read request from the local virtual hard disk VHD file under the trigger of the read request.
The method of claim 7, wherein the reading, by the server, the data corresponding to the read request from the local VHD file comprises:

Reading data corresponding to the read request from the difference disk;

And in the case that the data corresponding to the read request is not recorded in the difference disk, the data is read from a parent disk of the difference disk, wherein the data stored in the difference disk is the parent The changed data in the disk.
A data storage device, the device being applied to a server, comprising:

a receiving module configured to receive a write request for data;

The write module is configured to write the data to a sector in the local virtual hard disk VHD file and save it under the trigger of the write request.
A data reading device, the device being applied to a server, comprising:

a receiving module, configured to receive a read request for an operating system image file;

The reading module is configured to read data corresponding to the read request from the local virtual hard disk VHD file under the trigger of the read request.