CN113254268A

CN113254268A - Data processing method and device, equipment and storage medium

Info

Publication number: CN113254268A
Application number: CN202110552937.XA
Authority: CN
Inventors: 李凡
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-05-20
Filing date: 2021-05-20
Publication date: 2021-08-13

Abstract

The application discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring the target number N of backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N; based on the N backup tasks, backing up the target data stored in the at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk, wherein the backup rate is higher.

Description

Data processing method and device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of data processing, and relates to but is not limited to a data processing method, a data processing device, data processing equipment and a data processing storage medium.

Background

With the continuous development of virtualization technology, virtual machines have been widely applied to computer technology. The backup of a Virtual Machine (VM) can improve the reliability and stability of the VM, so that the method has very important significance.

Virtual machine backup, namely a process in which a virtual machine reads target data from a target file of a Virtual Disk (VDISK) through a backup task (backup job) and writes the target data into a backup image (backup image).

For a backup task, the virtual machine reads the data of a data block from a target file of the virtual disk, writes the data of the data block into the backup image, reads the data of the next data block and writes the data into the backup image, sequentially traverses all the data blocks of the target file, and writes all the target data into the backup image.

The existing virtual machine backup scheme is as follows: setting the number of the backup tasks as a fixed value N, reading data of N data blocks from target data in parallel by the virtual machine according to the N backup tasks, writing the data of the N data blocks into a backup mirror image, traversing all the target data in sequence, and completing backup.

However, in the above scheme, the backup rate cannot be guaranteed by using a fixed number of backup tasks to backup the target file.

Disclosure of Invention

The embodiment of the application provides a data processing method, a data processing device, equipment and a storage medium, and the data processing device has a high backup rate.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a data processing method, which comprises the following steps:

acquiring the target number N of backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N;

and backing up the target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

An embodiment of the present application provides a data processing apparatus, the apparatus includes:

the acquisition module is used for acquiring the target number N of the backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N;

and the backup module is used for backing up the target data stored in the at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

An embodiment of the present application further provides an electronic device, including: a memory storing a computer program operable on a processor and a processor implementing the above data processing method when executing the program.

The embodiment of the application also provides a storage medium, wherein a computer program is stored on the storage medium, and the computer program realizes the data processing method when being executed by a processor.

The data processing method, device, equipment and storage medium provided by the embodiment of the application comprise the following steps: acquiring the target number N of backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N; and backing up the target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks. In the scheme of the application, the target number N of the backup tasks is obtained first, and then the target data is backed up based on the N backup tasks. Therefore, the backup rate of the N backup tasks is higher than that of other backup tasks, so that the scheme of the application has higher backup rate in the backup process.

Drawings

FIG. 1 is a schematic diagram of an alternative configuration of a data processing system according to an embodiment of the present application;

fig. 2 is an alternative flow chart of a data processing method according to an embodiment of the present disclosure;

fig. 3 is an alternative flow chart of a data processing method according to an embodiment of the present application;

fig. 4 is an alternative flow chart of the data processing method according to the embodiment of the present application;

fig. 5 is an alternative flow chart of the data processing method according to the embodiment of the present application;

fig. 6A is an alternative schematic flow chart of a data processing method according to an embodiment of the present disclosure;

fig. 6B is an alternative schematic flow chart of the data processing method according to the embodiment of the present application;

fig. 7 is an alternative flowchart of a data processing method according to an embodiment of the present application;

fig. 8 is an alternative structural diagram of a backup process according to an embodiment of the present disclosure;

fig. 9 is an alternative structural diagram of a backup process according to an embodiment of the present application;

fig. 10 is an alternative structural diagram of a backup process provided in an embodiment of the present application;

fig. 11 is an alternative schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 12 is an alternative structural schematic diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will describe the specific technical solutions of the present application in further detail with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.

The embodiment of the application can provide a data processing method, a data processing device, data processing equipment and a storage medium. In practical applications, the data processing method may be implemented by a data processing apparatus, and each functional entity in the data processing apparatus may be cooperatively implemented by hardware resources of an electronic device (e.g., a terminal device), such as computing resources like a processor, and communication resources (e.g., for supporting communications in various manners like optical cables and cellular).

The data processing method provided by the embodiment of the application is applied to a data processing system, and the data processing system is composed of a client and a data processing end.

As an example, a data processing system may be configured as shown in FIG. 1, including: a client 10 and a data processing terminal 20.

In an example, the client 10 and the data processing side 20 may be the same physical entity; in one example, as shown in fig. 1, the client 10 and the data processing side 20 may be different physical entities, and the client 10 and the data processing side 20 interact with each other through the network 30.

Here, the client 10 is configured to receive an operation by a user, and send a backup request to the data processing terminal 20 based on the operation by the user. The data processing end 20 is configured to receive a backup request sent by the client 10 to backup target data.

In the embodiment of the present application, based on the data processing system shown in fig. 1, a client sends an instruction for backing up data to a data processing end, and the data processing end receives the instruction for backing up data and executes: acquiring the target number N of backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N; and backing up the target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks.

Embodiments of a data processing method, a data processing apparatus, a data processing device, and a storage medium according to the embodiments of the present application are described below with reference to a schematic diagram of a data processing system shown in fig. 1.

The present embodiment provides a data processing method, which is applied to a data processing apparatus, wherein the data processing apparatus can be implemented on an electronic device as a data processing end. The functions implemented by the method can be implemented by calling program code by a processor in an electronic device, and the program code can be stored in a computer storage medium.

The electronic device may be any device with information processing capability, and in one embodiment, the electronic device may be an intelligent terminal, for example, an electronic device with wireless communication capability such as a notebook, an AR/VR device, or a mobile terminal. In another embodiment, the electronic device may also be a computing-capable terminal device that is not mobile, such as a desktop computer, or the like.

Of course, the embodiments of the present application are not limited to the provided method and hardware, and may be implemented in various ways, for example, as a storage medium (storing instructions for executing the data processing method provided by the embodiments of the present application).

Fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application, which is used for backing up a target file in at least one virtual disk, and the backup process is now described in detail.

As shown in fig. 2, the method comprises the steps of:

s201, the electronic equipment obtains the target number N of the backup tasks.

The backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; m is different from N. In short, the backup rate corresponding to the target number N of backup tasks is optimal.

The backup rate is the data size of the data quantity of the virtual disk backed up to the backup image in unit time.

In one example, the optimal backup rate is related to the storage software type. Specifically, for a known storage type, the backup rate of the storage type is known under different numbers of backup tasks, that is, the optimal number of backup tasks corresponding to the optimal backup rate is known. That is, for the known storage type, the backup rate of the optimal number of backup tasks is higher than the backup rate of the other non-optimal number of backup tasks.

In this case, S201 may be implemented as: the electronic equipment acquires the type of the storage software, searches the backup task number corresponding to the optimal backup rate in the plurality of backup task numbers under the type of the storage software, and takes the backup task number as the target number N of the backup tasks.

In another example, there are more factors associated with the optimal backup rate and the specific factors that affect the optimal backup rate cannot be determined. In this case, S201 may be implemented as: the electronic equipment sets a plurality of numbers, and for the backup rate of each backup task in the plurality of numbers, the number of the backup tasks with the maximum backup rate in the plurality of backup rates is used as the target number N of the backup tasks.

S202, backing up target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk by the electronic device based on the N backup tasks.

The backup image is used for storing backup data. Specifically, for each virtual disk, a backup image may be established for storing backup data of the virtual disk.

The target data is part or all of the data to be backed up. In an embodiment of the present application, at least one virtual disk stores target data.

The electronic equipment configures the number of the backup tasks to be N, parallelly starts the N backup tasks through a backup manager (backup manager), and backs up target data of at least one virtual disk by adopting the N backup tasks. Specifically, the electronic device reads N data blocks of target data from at least one virtual disk, stores the N data blocks in a backup image corresponding to each virtual disk in the at least one virtual disk, then reads another N data blocks of the target data from the at least one virtual disk, stores the other N data blocks in a backup image corresponding to each virtual disk in the at least one virtual disk, and repeats reading and storing until all the target data are stored in the corresponding backup images, thereby completing backup of the target data.

It should be noted that, if the test data does not belong to the data to be backed up, the target data stored in the at least one virtual disk is all the data to be backed up; and if the test data belong to the data to be backed up, the target data stored in the at least one virtual disk is the data except the test data in the data to be backed up.

According to the data processing method provided by the embodiment of the application, the target number N of the backup tasks is obtained; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; said M is different from said N; and backing up the target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks. In the scheme of the application, the target number N of the backup tasks is obtained first, and then the target data is backed up based on the N backup tasks. Therefore, the backup rate of the N backup tasks is higher than that of other backup tasks, so that the scheme of the application has higher backup rate in the backup process.

The following describes a specific implementation of S201 in which the electronic device obtains the target number N of backup tasks. The method for obtaining N may include, but is not limited to, the following method a or method B.

As shown in fig. 3, the method a may include, but is not limited to, the following S2011A to S2013A.

S2011A, the electronic device sets at least two test quantities.

The number of the test quantity can be set by the electronic equipment according to actual requirements.

S2011A may be implemented as: the electronic device may set at least two test quantities according to a preset algorithm.

The preset algorithm for setting the test quantity may include: a random algorithm, an arithmetic of arithmetic increments of arithmetic differences, an arithmetic of arithmetic increments of geometric ratios, etc.

For example, the electronic device may set 10 test quantities, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, respectively, according to an arithmetic of.

S2012A, the electronic device respectively counts the backup rate of each backup task with the test quantity in the at least two test quantities.

Specifically, the counting, by the electronic device, the backup rate under the first number of backup tasks includes: and starting a first number of backup tasks in parallel, performing backup on the test data by adopting the first number of backup tasks, and counting the backup rate when the test data is backed up by adopting the first number of backup tasks.

The first number is any one of at least two test numbers.

The electronic device traverses each test quantity of the at least two test quantities, and for each test quantity, the electronic device refers to the execution process of the first quantity to perform operation, so as to obtain a backup rate corresponding to each test quantity.

In one example, the test data may be a set of known data dedicated to determining the target number of tasks N. In this case, the test data does not belong to the data to be backed up.

In another example, the test data belongs to data to be backed up.

The backing up of the test data by the electronic device using the first number of backup tasks may include: the electronic equipment reads the first number of data blocks from the initial segment of the test data, stores the first number of data blocks into the backup mirror image, then reads the next first number of data blocks of the test data, stores the next first number of data blocks into the backup mirror image, and repeatedly reads and stores the data blocks.

The electronic device may count the backup rate when the test data is backed up using the first number of backup tasks, where the counting includes: the electronic equipment counts the data volume backed up in the first time period, and the quotient of the data volume and the duration of the first time period is used as the backup rate of the first number of backup tasks.

Or the electronic equipment adopts a speed measurement algorithm to measure the real-time backup rate under the first number of backup tasks in real time, and the electronic equipment calculates the average value of all the real-time backup rates in a period of time to be used as the backup rate under the first number of backup tasks.

For example, the test quantities include: 2. 4, 6, 8, 10, 12, 14, 16, 18, 20; the electronic equipment adopts 2 backup tasks to backup test data, and the backup rate under the 2 backup tasks is counted to be 20 megabytes per second (MB/s); the electronic equipment adopts 4 backup tasks to backup test data, and the backup rate under the 4 backup tasks is counted to be 30 MB/s; the electronic equipment adopts 6 backup tasks to backup test data, and the backup rate under the 6 backup tasks is counted to be 40 MB/s; the electronic equipment adopts 8 backup tasks to backup test data, and the backup rate under the 8 backup tasks is counted to be 50 MB/s; the electronic equipment adopts 10 backup tasks to backup test data, and the backup rate under the 10 backup tasks is counted to be 60 MB/s; the electronic equipment adopts 12 backup tasks to backup test data, and the backup rate under the 12 backup tasks is counted to be 65 MB/s; the electronic equipment adopts 14 backup tasks to backup test data, and the backup rate under the 14 backup tasks is counted to be 75 MB/s; the electronic equipment adopts 16 backup tasks to backup test data, and the backup rate under the 16 backup tasks is counted to be 70 MB/s; the electronic equipment adopts 18 backup tasks to backup test data, and the backup rate under the 18 backup tasks is counted to be 60 MB/s; the electronic equipment adopts 20 backup tasks to backup the test data, and the backup rate under the 20 backup tasks is counted to be 60 MB/s.

For example, the test number of the backup tasks and the backup rate under the test number of the backup tasks may be saved as table 1.

TABLE 1

Number of tests	Testing backup rate under a number of backup tasks
		2	20MB/s
4	30MB/s
		6	40MB/s
8	50MB/s
		10	60MB/s
12	65MB/s
		14	75MB/s
16	70MB/s
		18	60MB/s
20	60MB/s

S2013A, the electronic device uses the test quantity with the largest backup rate from among the at least two test quantities as the N.

The electronic device compares the at least two backup rates obtained in S2012A to obtain a maximum backup rate of the at least two backup rates, checks the test quantity corresponding to the maximum backup rate, and takes the test quantity corresponding to the maximum backup rate as the target quantity N of the backup tasks.

For example, assuming that the backup rate obtained in S2012A for each test number of backup tasks is shown in table 1, S2013A may be implemented as: and the electronic equipment reads the table 1, obtains the maximum backup rate of 75MB/s, finds that the test quantity corresponding to the backup rate of 75MB/s is 14, and considers 14 as the target quantity N of the backup tasks.

As shown in fig. 4, the method B may include, but is not limited to, the following steps S2011B to S2017B.

It should be noted that the following initial measurement number and the incremented measurement number are relative concepts. That is, for one cycle described below, there is a pair of the initial measurement number and the incremented measurement number.

S2011B, the electronic device obtains an initial test quantity.

Wherein the initial test number is greater than or equal to 1.

S2011B may be implemented as: the electronic equipment acquires the default number of the equipment as the initial measurement number; or the electronic equipment selects one quantity from a plurality of preset test quantities as an initial test quantity based on the operation of a user; alternatively, the electronic device may use the number assigned in other manners as the initial test number.

S2012B, the electronic device counts the backup rate under the initial test number of backup tasks to obtain an initial rate.

For the specific implementation of S2012B, refer to the implementation process of the electronic device in S2012A for counting the backup rate under the first number of backup tasks, which is not described herein again.

And S2013B, the electronic equipment increases the initial test quantity to the increased test quantity according to an increasing rule.

The specific content of the increment rule can be configured according to the actual requirement, which is not limited in the embodiment of the present application.

For example, the increment rule may include: the arithmetic difference is increased by 2.

As another example, the increment rule may include: the equal ratio is increased, and the ratio is 2.

Specifically, S2013B may be implemented as: and the electronic equipment carries out incremental calculation on the initial measurement quantity according to the limiting conditions of the incremental rule to obtain the incremental measurement quantity.

For example, assuming that the initial measurement number is 2, the increment rule includes equal difference increment, and the difference is 2, the electronic device calculates 2 plus 2 to be equal to 4 after the equal difference increment with the difference of 2 is performed on the initial measurement number; i.e. an incremental number of tests of 4 is obtained.

And S2014B, the electronic equipment counts the backup rate under the incremental test number of backup tasks to obtain the incremental rate.

For specific implementation of S2014B, reference may be made to an implementation process of the electronic device in S2012A for counting the backup rate under the first number of backup tasks, which is not described herein again.

S2015B, the electronic device determines whether the incremented rate is less than or equal to the initial rate.

The electronic device determines whether the incremented rate is less than or equal to the initial rate, and if the incremented rate is less than or equal to the initial rate, performs S2016B; if the incremented rate is greater than the initial rate, the following S2017B is performed.

S2016B, the electronic equipment takes the initial measurement number corresponding to the initial rate as the N.

And the electronic equipment takes the initial measurement quantity corresponding to the initial rate in the cycle as the target quantity N of the backup tasks.

And S2017B, the electronic equipment takes the increased test quantity as a new initial test quantity.

And the electronic equipment takes the increased test quantity as a new initial test quantity, and continuously counts the backup rate under the backup task of the new initial test quantity until the increased rate is less than or equal to the initial rate. In other words, the electronic device takes the incremented test number as a new initial test number, and executes the above-mentioned S2011B to S2017B again in sequence.

As an example, as shown in fig. 5, implementation of mode B may include, but is not limited to, the following S501 to S510.

S501, the electronic equipment obtains a first test quantity.

S502, the electronic equipment counts the backup rate under the first test number of backup tasks to be used as a first rate.

S503, the electronic device increases the first test quantity to a second test quantity.

S504, the electronic device counts the backup rate under the second test number of backup tasks as a second rate.

S505, the electronic equipment judges whether the second rate is less than or equal to the first rate.

If the electronic device determines that the second rate is less than or equal to the first rate, the following S506 is performed.

If the electronic device determines that the second rate is greater than the first rate, the following S507 is executed.

S506, the electronic equipment takes the first rate as N.

And S507, the electronic equipment increases the second testing number to a third testing number.

And S508, the electronic equipment counts the backup rate under the backup tasks with the third test quantity to serve as a third rate.

S509, the electronic device judges whether the third rate is less than or equal to the second rate.

If the electronic device determines that the third rate is less than or equal to the second rate, the following step S510 is performed.

If the electronic device determines that the third rate is greater than the second rate, referring to S507 to S510, performing a loop operation; until N is obtained.

And S510, the electronic equipment takes the second rate as N.

Specific implementations of S202 may include, but are not limited to, scheme a, scheme b, or scheme c.

If only one virtual disk (the first virtual disk) includes the target data, S202 may be implemented as the scheme a, and if P virtual disks include the target data, S202 may be implemented as the scheme b. Wherein P is greater than 1.

And according to the scheme a, the electronic equipment backs up the target data stored in the virtual disk to the backup image corresponding to the virtual disk based on the N backup tasks.

Specifically, the electronic device starts N backup tasks in parallel through the backup manager, and backs up the target data of the first virtual disk by using the N backup tasks.

The process of the electronic device for backing up the target data of the first virtual disk by adopting the N backup tasks comprises the following steps: the electronic equipment reads N data blocks of target data from the first virtual disk through N backup tasks, stores the N data blocks into a backup image corresponding to the first virtual disk, reads another N data blocks of the target data from the first virtual disk, stores the other N data blocks into the backup image corresponding to the first virtual disk, and repeats reading and storing until all the target data in the first virtual disk are stored into the backup image corresponding to the first virtual disk.

Scheme b as shown in fig. 6A may include, but is not limited to, the following S2021b to S2022 b.

S2021b, the electronic device allocates the N backup tasks to the P virtual disks.

The electronic equipment distributes the N backup tasks to the P virtual disks according to a distribution rule.

The specific content of the distribution rule may be configured according to actual requirements, which is not limited in the embodiment of the present application. For example, the allocation rules may include: an average assignment, a random assignment, or a priority assignment of virtual disks.

Assuming that the allocation rule is an average allocation, the target number of backup tasks obtained in S201 is 20, which are: backup task 1, backup task 2 … … backup task 20; the 5 virtual disks comprise target data, namely a virtual disk 1 and a virtual disk 2 … … virtual disk 5; the electronic device may allocate 20 backup tasks to 5 virtual disks on average according to the average allocation rule, that is, each virtual disk corresponds to 4 backup tasks, as shown in table 2.

TABLE 2

Virtual disk	Backup task corresponding to virtual disk
		Virtual disk 1	Backup task 1, backup task 2, backup task 3, and backup task 4
Virtual disk 2	Backup task 5, backup task 6, backup task 7, and backup task 8
		Virtual disk 3	Backup task 9, backup task 10, backup task 11, and backup task 12
Virtual disk4	Backup task 13, backup task 14, backup task 15, and backup task 16
		Virtual disk 5	Backup task 17, backup task 18, backup task 19, and backup task 20

S2022b, for each virtual disk in the P virtual disks, the electronic device backs up the target data stored in the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk.

Specifically, the electronic device executes the following operations for each virtual disk in the P virtual disks until all the target data in the P virtual disks are backed up. Taking the second virtual disk as an example, the executed operations include: the electronic equipment starts a backup task corresponding to the second virtual disk in parallel through the backup manager, and the backup task corresponding to the second virtual disk is adopted to backup target data of the second virtual disk.

In the embodiment of the present invention, the specific implementation that the electronic device backs up the target data of the second virtual disk by using the backup task corresponding to the second virtual disk may refer to scheme a, where details are not repeated here, and the electronic device backs up the target data of the first virtual disk by using N backup tasks.

It can be seen that, in the scheme b, when backing up the target data, the backup task corresponding to each virtual disk is fixed.

The implementation of scheme c is explained below.

In the scheme c, when backing up the target data, the backup task corresponding to each virtual disk may be dynamically adjusted. For example, after the at least one virtual disk completes the backup, the backup task corresponding to the virtual disk whose backup is not completed may be dynamically adjusted. As shown in fig. 6B, implementation of scheme c may include, but is not limited to, S2021c through S2024c described below.

S2021c, the electronic device allocates the N backup tasks to the P virtual disks.

Reference may be made to S2021b for specific implementation of S2021 c.

S2022c, for each virtual disk in the P virtual disks, the electronic device backs up the target data stored in the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk.

The implementation of S2022c may refer to S2022b, and unlike S2022b, in the scheme b, S2022b executes until the backup of all virtual disks is completed. In scenario c, S2022c is executed, and the electronic device executes S2023c after executing the first time.

Wherein the first time may be a preset time period. The duration of the first time is not limited, and can be configured according to the time requirement. For example, the first time may be set to a smaller time period corresponding to the execution of S2023c immediately after the execution of S2022c, i.e., the detection in S2023c is in real time.

S2023c, the electronic device detects the backup states of the P virtual disks.

Wherein, the backup state comprises: non-backup, backup-neutralization and backup are completed.

The electronic device may detect the backup status of the virtual disk by using a detection algorithm, and the implementing of S2023c may include: the electronic equipment detects the backup states of the P virtual disks by adopting a detection algorithm.

Assuming that P is 5, the P virtual disks are respectively a virtual disk 1, a virtual disk 2, a virtual disk 3, a virtual disk 4 and a virtual disk 5; the backup states of the 5 virtual disks detected by the electronic device can be as shown in table 3.

TABLE 3

S2024c, the electronic device allocates the N backup tasks to the P-K virtual disks.

And the P-K virtual disks are virtual disks which are not finished with backup in the P virtual disks.

For example, the backup states of the P virtual disks acquired in S2023c are shown in table 3, and the backup task corresponding to each virtual disk is shown in table 2, then the electronic device evenly allocates the backup task 17, the backup task 18, the backup task 19, and the backup task 20 corresponding to the virtual disk 5 to the virtual disk 1, the virtual disk 2, the virtual disk 3, and the virtual disk 4; the backup task corresponding to the reallocated virtual disk may be as shown in table 4.

TABLE 4

Where,/means no backup task.

S2025c, for each virtual disk of the P-K virtual disks, the electronic device backs up the target data stored in the virtual disk to the backup image corresponding to the virtual disk based on the backup task allocated to the virtual disk.

For the specific implementation of S2025c, reference may be made to S2022b, which is not described again.

It can be understood that the electronic device may detect the backup progress of each virtual disk in real time, and when it is detected that the third virtual disk completes the backup, obtain at least one backup task corresponding to the third virtual disk, and allocate the at least one backup task corresponding to the third virtual disk to the fourth virtual disk.

And the fourth virtual disk is a virtual disk with incomplete backup. In an example, the fourth virtual disk is any one of the incomplete backed-up virtual disks. In another example, the fourth virtual disk is a virtual disk with the highest priority level among any virtual disks that do not complete backup. In yet another example, the fourth virtual disk is a plurality of outstanding backup virtual disks.

The data processing method provided in the embodiment of the present application is further described below by using a specific application scenario.

The number of reads and writes per second (IOPS) is used as an index to measure the performance of the storage device.

Input Output (IO) depth, which may refer to the number of IOs currently concurrently committed to a storage device. The IO depth may also be referred to as IO queue depth, which is equal to the number of backup tasks in this embodiment of the present application. For example, 1 IO is submitted to the storage device, and before the IO returns, 3 IOs are submitted at the same time, and the IO depth of the storage device is 4. Assuming that 1 IO is submitted to the storage device, after the IO returns, the next 1 IO can be submitted, and the IO depth of the storage device is 1.

The optimal IO depth may refer to an IO depth corresponding to the highest IOPS performance.

The IO performance of the currently mainstream storage device has such a characteristic that the IO performance increases with the increase of the IO depth, and the IOPS starts to decrease after reaching a certain value, for example, table 6 shows the IO performance of a storage device, and it can be seen that the optimal IO depth of the storage device is 32.

TABLE 6

IO depth	1	2	4	8	16	32	64	128
									IOPS	4235	6233	8568	10531	11921	12756	12168	12119

In practical applications, the optimal IO depths corresponding to different types of storage devices may be different, for example, the optimal IO depth of a NAS type storage device may be 16, the optimal IO depth of a SAN type storage device may be 32, and the optimal IO depth of a distributed virtual storage type storage device may be 64.

The IO depth control for virtual machine backup may include the following scheme one or scheme two.

The first scheme is as follows: 1 VDISK is corresponding to and fixed 1 backup joba, namely the backup IO depth is fixed to 1. Under the scheme, the IOPS performance of the storage device is poor, the backup time is long, and the backup speed is low.

Scheme II: and the 1 VDISK corresponds to the fixed X backup jobs, namely the backup IO depth is fixed to X. Because the optimal IO depths of various storage devices are different, the backup rate is low because the fixed X value is difficult to correspond to the optimal IO depth.

The VM backup rate is low under the two schemes, and the scheme provided by the invention has high backup rate.

Suppose the VM includes a virtual disk a, and the target data in the virtual disk a needs to be backed up. As shown in fig. 7, the data processing procedure may include S701 to S712 described below.

S701, the electronic equipment obtains the number 4 of default backup tasks.

S702, the electronic device starts 4 backup tasks through the backup manager, and stores the target data in the virtual disk A into the backup image.

And S703, counting the average backup rate of 4 backup tasks in 20 seconds by the electronic equipment through the backup manager.

The average backup rate of the electronic equipment under 4 backup tasks is 30MB/s within 20 seconds through the backup manager.

For example, fig. 8 illustrates backup data flow under 4 backup tasks.

S704, the electronic device adds 2 backup tasks to the virtual disk A through the backup manager.

S705, the electronic device runs 6 backup tasks through the backup manager, and continues to backup the target data in the virtual disk a.

And S706, counting the average backup rate of 6 backup tasks in 20 seconds by the electronic equipment through the backup manager.

The average backup rate of the electronic equipment under 6 backup tasks is counted by the backup manager to be 50MB/s within 20 seconds.

For example, fig. 9 illustrates backup data flow under 6 backup tasks.

And S707, the electronic equipment determines that the backup rate is not reduced through comparison of the backup manager.

The average backup rate of the electronic equipment under the condition that the backup manager compares 6 backup tasks is larger than the average backup rate under 4 backup tasks.

S708, the electronic device adds 2 backup tasks to the virtual disk A through the backup manager.

And S709, the electronic equipment runs 8 backup tasks through the backup manager and continuously backs up the target data in the virtual disk A.

And S710, counting the average backup rate of 8 backup tasks in 20 seconds by the electronic equipment through the backup manager.

The average backup rate of the electronic equipment under 8 backup tasks is 45MB/s within 20 seconds through the backup manager.

For example, fig. 10 illustrates backup data flow under 8 backup tasks.

And S711, the electronic equipment determines that the backup rate is reduced through comparison of the backup manager.

And comparing that the average backup rate of 45MB/s under 8 backup tasks is less than the average backup rate of 50MB/s under 6 backup tasks by the electronic equipment through the backup manager.

And S712, the electronic device runs 6 backup tasks through the backup manager, and continues to backup the target data in the virtual disk A until the backup is completed.

Compared with the traditional scheme in the industry, in the data processing method, the number of the backup tasks can be continuously and dynamically adjusted in the backup process, and the backup rate is monitored to judge whether the optimal IO depth of the storage device is reached. Finally, the number of VM backup tasks can be adjusted in a self-adaptive mode, so that the optimal IO depth of the storage device is achieved, and the VM backup rate is optimal. Therefore, the backup method improves the backup rate and shortens the backup time.

Fig. 11 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, and as shown in fig. 11, the data processing apparatus 110 may include an obtaining module 1101 and a backup module 1102. Wherein:

an obtaining module 1101, configured to obtain a target number N of backup tasks; the backup rate under the N backup tasks is greater than or equal to the backup rate under the M backup tasks; the M is different from the N.

The backup module 1102 is configured to backup, based on the N backup tasks, target data stored in at least one virtual disk to a backup image corresponding to each virtual disk in the at least one virtual disk.

In some embodiments, the obtaining module 1101 is specifically configured to:

setting at least two test quantities;

respectively counting the backup rate of each backup task with the test quantity in the at least two test quantities;

and taking the test number with the maximum corresponding backup rate in the at least two test numbers as the N.

In some embodiments, the obtaining module 1101 is specifically configured to obtain an initial test number;

counting the backup rate under the initial test quantity of backup tasks to obtain an initial rate;

increasing the initial test quantity to the increased test quantity according to an increasing rule;

counting the backup rate of the incremental test quantity of backup tasks to obtain the incremental rate;

if the increased speed is greater than the initial speed, taking the increased test quantity as a new initial test quantity, and continuously counting the backup speed under the backup tasks with the new initial test quantity until the increased speed is less than or equal to the initial speed;

and taking the initial measurement number corresponding to the initial rate as the N when the increased rate is less than or equal to the initial rate.

In some embodiments, if the at least one virtual disk includes one virtual disk, the backup module 1102 is specifically configured to:

and backing up the target data stored in the virtual disk to a backup image corresponding to the virtual disk based on the N backup tasks.

In some embodiments, if the at least one virtual disk includes P virtual disks, where P is greater than 1, the backup module 1102 is specifically configured to:

distributing the N backup tasks to the P virtual disks;

and for each virtual disk in the P virtual disks, backing up the target data stored in the virtual disk to a backup image corresponding to the virtual disk based on the backup task distributed to the virtual disk.

In some embodiments, the backup module 1102 is further configured to:

if the K virtual disks included in the P virtual disks are backed up, distributing the N backup tasks to the P-K virtual disks; the P-K virtual disks are virtual disks which are not backed up in the P virtual disks;

and for each virtual disk in the P-K virtual disks, backing up the target data stored in the virtual disk to a backup image corresponding to the virtual disk based on the backup task distributed to the virtual disk.

It should be noted that the data processing apparatus provided in the embodiment of the present application includes each included unit, and may be implemented by a processor in an electronic device; of course, the implementation can also be realized through a specific logic circuit; in the implementation process, the Processor may be a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.

The above description of the apparatus embodiments, similar to the above description of the method embodiments, has similar beneficial effects as the method embodiments. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the data processing method is implemented in the form of a software functional module and sold or used as a standalone product, the data processing method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the related art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, an embodiment of the present application provides an electronic device, which includes a memory and a processor, where the memory stores a computer program that can be run on the processor, and the processor executes the computer program to implement the steps in the data processing method provided in the foregoing embodiment.

Accordingly, embodiments of the present application provide a storage medium, that is, a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the data processing method provided in the above embodiments.

Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that fig. 12 is a schematic hardware entity diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 12, the electronic device 120 includes: a processor 1201, at least one communication bus 1202, a user interface 1203, at least one external communication interface 1204 and a memory 1205. Wherein the communication bus 1202 is configured to enable connective communication between such components. The user interface 1203 may include a display screen, and the external communication interface 1204 may include a standard wired interface and a wireless interface, among others.

The Memory 1205 is configured to store instructions and applications executable by the processor 1201, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by the processor 1201 and modules in the electronic device, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in some embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the related art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only for the embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of data processing, the method comprising:

2. The method of claim 1, wherein obtaining the target number N of backup tasks comprises:

setting at least two test quantities;

3. The method of claim 1, wherein obtaining the target number N of backup tasks comprises:

acquiring an initial test quantity;

4. The method according to claim 1, wherein if the at least one virtual disk includes one virtual disk, the backing up the target data stored in the at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks includes:

5. The method of claim 1, wherein if the at least one virtual disk comprises P virtual disks, P is greater than 1; the backing up the target data stored in the at least one virtual disk to the backup image corresponding to each virtual disk in the at least one virtual disk based on the N backup tasks includes:

distributing the N backup tasks to the P virtual disks;

6. The method of claim 5, further comprising:

7. A data processing apparatus, characterized in that the apparatus comprises:

8. An electronic device comprising a memory and a processor, the memory storing a computer program operable on the processor, the processor implementing the data processing method of any one of claims 1 to 6 when executing the program.

9. A storage medium on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the data processing method of any one of claims 1 to 6.