WO2015090113A1 - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
WO2015090113A1
WO2015090113A1 PCT/CN2014/089533 CN2014089533W WO2015090113A1 WO 2015090113 A1 WO2015090113 A1 WO 2015090113A1 CN 2014089533 W CN2014089533 W CN 2014089533W WO 2015090113 A1 WO2015090113 A1 WO 2015090113A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
storage unit
volatile storage
log record
size
Prior art date
Application number
PCT/CN2014/089533
Other languages
French (fr)
Chinese (zh)
Inventor
任努努
金添福
李明君
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2015090113A1 publication Critical patent/WO2015090113A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory

Definitions

  • the present invention relates to the field of cloud computing, and in particular, to a data processing method and device.
  • Virtualization technology is one of the key technologies in the field of cloud computing.
  • the main principle is to virtualize the physical resources of a physical machine into multiple virtual machines that can run the operating system independently. Each virtual machine can independently implement the physical machine. The function does not interfere with each other.
  • the virtual machine usage process is consistent with the physical machine.
  • the virtual machine can be considered as an abstract form of the physical machine.
  • FIG. 1 is a schematic diagram of a prior art physical machine virtualization.
  • various physical resources on the hardware platform 100 including the central processing unit 102, the memory 103, the storage 104 (disk, solid state drive (SSD), etc.), the network card 105) are virtualized by the management system 110.
  • the abstraction is a plurality of virtual machines (VMs) 130, exemplified by 130a, 130b, and 130c, wherein the physical storage 104 can be local or remote (accessed through the storage network when located remotely).
  • the function of the virtual resource is exactly the same as that of the real physical resource.
  • the operation of the virtual resource in the virtual machine is completely consistent with the physical machine.
  • a separate operating system 132, as well as one or more applications 131, can be installed in different virtual machines.
  • a virtual hardware platform may be included in the virtual machine 130.
  • the virtual machine management system 110 includes a plurality of software components, including a Virtual Machine Monitor (VMM) 120, exemplified by 120a, 120b, and 120c in the figure; managing virtualized software components, and the like.
  • VMM Virtual Machine Monitor
  • the management system is responsible for abstracting the physical storage 104 into separate virtual storages 124 for use by different virtual machines 130. All virtual machines 130 use separate virtual storages 124 on their surface, in effect one or more of them. Different spaces in physical storage 104 (Use this file or other form to occupy this part of the space).
  • the management system 110 maps the request for the virtual storage 124 to a request for a different space of the physical storage 104, thereby enabling the different virtual machines 130 to have their own independent virtual storages 124, Independent of each other in use.
  • the write request data is first written to the high-speed, large-capacity, volatile main storage medium 103 (memory), and is synchronously written to the high-speed, small-capacity, non-volatile random access memory 104 (NVRAM) and returned. Since the NVRAM capacity is usually small, write failure may occur.
  • Embodiments of the present invention provide a data processing method and device, which can reduce the probability of failure to write data to NVRAM.
  • a method of data processing is provided, the method being applied to a data processing device, the data processing device comprising a control unit, a first non-volatile storage unit and a second non-volatile storage unit, The speed at which the first non-volatile memory unit writes data is higher than the speed at which the second non-volatile memory unit writes data, the method comprising:
  • the control unit receives a write data request from a virtual machine, the write data request including first data
  • the control unit controls the first non-volatile storage unit to dump a log record of the second data in the first non-volatile storage unit to the second non-volatile storage unit So that the remaining space of the first non-volatile memory unit is increased.
  • the method before the comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit, the method further Includes:
  • the IO control module of the control unit sends a log record size of the first data to an IO statistic module of the control unit, where the IO statistic module saves a remaining space of the first nonvolatile storage unit;
  • the comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
  • the IO statistic module compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit, and returns comparison result information to the IO control module.
  • the method further includes:
  • the IO statistics module compares the size of the log record of the first data again. And a remaining space of the first nonvolatile storage unit.
  • the method before the comparing the size of the log record of the first data with the remaining space of the first nonvolatile storage unit, the method further include:
  • the IO control module of the control unit sends query request information to the IO statistics module of the control unit, where the query request information is used to query the remaining space of the first non-volatile storage unit;
  • the IO control module acquires a remaining space of the first non-volatile storage unit from the IO statistic module;
  • the comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
  • the IO control module compares a size of the log record of the first data with a remaining space of the first non-volatile storage unit.
  • the method further includes:
  • the IO control module sends the query request to the IO statistic module again. information.
  • the logging of the first data The size of the first non-volatile storage unit is larger than the remaining space of the first non-volatile storage unit, and the control unit controls the first non-volatile storage unit to dump the second data in the first non-volatile storage unit
  • the second non-volatile storage unit includes:
  • the IO statistic module sends data dump indication information to the IO control module, where the data dump indication information includes: address information of the second data;
  • the IO control module generates data dump request information according to the address information of the second data, and sends the data dump request information to the first non-volatile storage unit to enable the first non-volatile
  • the storage unit dumps the log record of the second data to the second non-volatile storage unit according to the data dump request information, and receives the return of the second non-volatile storage unit After successfully dumping the information, the storage space occupied by the log record of the second data is released to increase the remaining space of the first non-volatile storage unit.
  • the method further includes:
  • the IO control module sends a size of the log record of the second data to the IO statistic module, and the IO statistic module updates the first non-volatile storage according to the size of the log record of the second data. a remaining space of the unit and the second non-volatile storage unit.
  • the method further includes:
  • the size of the log record of the first data is not greater than the remaining space of the first non-volatile storage unit, and the write speed of the first non-volatile storage unit is not greater than a preset threshold At the time of the value, the log record of the first data is written to the first non-volatile storage unit.
  • the logging of the first data is written to the first non-volatile storage After the unit, the method further includes:
  • the IO control module receives the successfully written information returned by the first non-volatile storage unit, and sends the size of the log record of the first data to the IO statistics module, according to the IO statistics module.
  • the size of the log record of the first data updates the remaining space of the first non-volatile storage unit.
  • the method further includes:
  • the control unit receives a read data request from the virtual machine, and the read data request includes a data identifier to be read out;
  • a data processing apparatus comprising: a control unit, a first non-volatile storage unit and a second non-volatile storage unit, the first non-volatile storage unit writing The speed of the data is higher than the speed at which the second non-volatile storage unit writes data;
  • the control unit is configured to receive a write data request from a virtual machine, where the write data request includes first data;
  • the first non-volatile storage unit is configured to dump the log record of the second data to The second nonvolatile memory unit;
  • the second non-volatile storage unit is configured to write a log record of the second data.
  • the control unit includes an IO control module and an IO statistics module;
  • the IO control module is configured to send a log record size of the first data to the IO statistic module;
  • the IO statistics module saves a remaining space of the first non-volatile storage unit, and compares a size of a log record of the first data sent by the IO control module with the first non-volatile storage unit. Remaining space and returning comparison result information to the IO control module.
  • the IO statistic module is further configured to: if a size of the log record of the first data is greater than that of the first non-volatile storage unit The remaining space, after delaying the specified threshold time, compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit again.
  • control unit includes an IO control module and an IO statistics module;
  • the IO control module is configured to send query request information to the IO statistic module, where the query request information is used to query a remaining space of the first non-volatile storage unit;
  • the IO control module is further configured to: if a size of the log record of the first data is greater than that of the first non-volatile storage unit After the remaining space is delayed by the specified threshold time, the query request information is sent to the IO statistic module again.
  • the IO statistic module is further configured to send data dump indication information to the IO control module,
  • the data dump indication information includes: address information of the second data;
  • the IO control module is further configured to generate data dump request information according to the address information of the second data sent by the IO statistics module, and send the data dump request information to the first non-volatile storage unit;
  • the second non-volatile storage unit is further configured to return information that is successfully dumped to the first non-volatile storage unit;
  • the first nonvolatile storage unit is further configured to dump the log record of the second data to the second nonvolatile storage unit according to the data dump request information sent by the IO control module After receiving the successfully dumped information returned by the second non-volatile storage unit, releasing the storage space occupied by the log record of the second data, so that the first non-volatile storage The remaining space of the unit increases.
  • the IO control module is further configured to be used according to the first possible implementation manner of the second aspect, or the second possible implementation manner of the second aspect
  • the successfully dumped information sends the size of the log record of the second data to the IO statistics module;
  • the IO statistic module is further configured to update, according to the size of the log record of the second data sent by the IO control module, the first non-volatile storage unit and the second non-volatile storage unit remaining space.
  • the device further includes: a writing unit, configured, if the size of the first data is recorded Writing not less than the remaining space of the first nonvolatile storage unit, and writing the log record of the first data when the write speed of the first nonvolatile storage unit is not greater than a preset threshold The first non-volatile storage unit.
  • the first non-volatile storage unit is further configured to return information that is successfully written to the IO control module;
  • the IO control module is further configured to receive information that is successfully written by the first non-volatile storage unit, and send a size of a log record of the first data to the IO statistics module;
  • the IO statistic module is further configured to update a remaining space of the first non-volatile storage unit according to a size of a log record of the first data sent by the IO control module.
  • control unit is further configured to use the virtual Receiving a read data request, the read data request including a data identifier to be read out;
  • the control unit receives a write data request from the virtual machine, the write data request includes the first data, determines the log record of the first data, and compares the size and the log record of the first data. a remaining space of a non-volatile storage unit, if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, the control unit controls the first non-volatile storage unit to The log record is dumped to the second non-volatile storage unit to increase the remaining space of the first non-volatile storage unit. Thereby, the probability of failure to write data to the first nonvolatile memory cell can be reduced.
  • FIG. 1 is a schematic diagram of a prior art physical machine virtualization
  • FIG. 2 is a schematic structural diagram of a data processing device according to Embodiment 1 of the present invention.
  • FIG. 3 is a flowchart of a method for data processing according to Embodiment 1 of the present invention.
  • FIG. 4 is a schematic structural diagram of a data processing device according to Embodiment 2 of the present invention.
  • FIG. 5 is a flowchart of a method for data processing according to Embodiment 2 of the present invention.
  • FIG. 6 is a flowchart of a method for data dumping according to Embodiment 3 of the present invention.
  • FIG. 7 is a schematic diagram of a data processing device according to Embodiment 4 of the present invention.
  • multiple virtual storages are located on the same physical storage medium. This causes that when multiple virtual machines send read and write requests to their respective virtual storages, they essentially initiate read and write requests to the same physical storage medium.
  • the physical storage medium has a number of technical indicators, and the number of times of reading/receiving operations per second (IOPS) determines the speed at which it responds to read and write requests. The larger the value, the faster the response.
  • IOPS reading/receiving operations per second
  • the specific IOPS value is related to the type of read and write requests.
  • the read and write request types can be divided into sequential read and write and random read and write.
  • Sequential read and write means that the read and write requests are continuous in the physical space of the storage medium
  • random read and write means that the read and write requests are discontinuous and discontinuous in the physical space of the storage medium.
  • the sequential read and write IOPS of physical storage media is much larger than random read and write IOPS.
  • Randomness In a virtualized environment, when multiple virtual machine read and write requests are sent to the same physical storage, it will generate a great deal. Randomness. Eventually, the read and write performance of each virtual storage is rapidly reduced due to mutual influence, which greatly affects the experience of the virtual machine user.
  • the large-capacity storage media used in the virtualized environment mainly include a memory, a disk, a solid state drive (SSD), a non-volatile random access memory (NVRAM), and the like. From the speed comparison, memory, NVRAM, etc. is the fastest IO response medium, in which the data will be lost after the power is lost, NVRAM will not, but NVRAM is expensive; in terms of capacity and cost, the disk is the largest and The lowest cost media, but with the lowest IOPS, especially random IOPS. These kinds of media have contradictions in speed, capacity and cost. In virtualized storage deployment, various storage media are generally matched according to user requirements, so that a balance is achieved in speed, capacity and cost.
  • FIG. 2 is a schematic structural diagram of a data processing device 20 according to Embodiment 1 of the present invention.
  • the data processing device 20 includes a memory 201, a control unit 202, a first non-volatile storage unit 203, and a second non-volatile storage unit 204.
  • the first non-volatile storage unit 203 writes data faster than the second non-volatile storage unit 204 writes data, thereby enabling fast backup of service data, especially for in-memory storage.
  • the processing device needs to store the service data, it can be guaranteed Store data security.
  • the memory 201 is a high-speed, large-capacity, volatile main memory, and needs to respond to read and write data requests.
  • the first non-volatile storage unit 203 is a high-speed, small-capacity, non-volatile cache medium, and is responsible for buffering write request data. And combining with the control unit 202 to perform data dumping; the second non-volatile storage unit 204 is a low-speed, large-capacity, non-volatile persistent medium, the first non-volatile storage unit 203 and the second non-easy
  • the dysfunctional storage unit 204 operates in conjunction to persist store write request data.
  • the read data request response When the read data request response is made, the response is performed by the memory 201; when the write data request response is made, the first nonvolatile storage unit 203 and the second nonvolatile storage unit 204 respond, and at the first non- When the remaining space of the volatile storage unit 203 is insufficient, the write request data is dumped into the second nonvolatile storage unit 204 in an asynchronous manner.
  • the read and write data request and data dump process are managed by the control unit 202 to achieve a high speed response to read and write data requests.
  • FIG. 3 is a flowchart of a method for data processing according to Embodiment 1 of the present invention, and the method is applied to the data processing device 20 shown in FIG. 2. As shown in FIG. 3, the method specifically includes:
  • the control unit receives a write data request from a virtual machine, where the write data request includes first data.
  • the virtual machine When the user writes data to the computer, the virtual machine issues a write data request to the control unit 202, the write data request including the first data to be written.
  • determining the log record of the first data according to the first data included in the write data request belongs to the prior art, and details are not described herein.
  • the control unit controls the first non-volatile storage unit to dump the log record of the second data in the first non-volatile storage unit to the second non-volatile
  • the storage unit is configured to increase the remaining space of the first nonvolatile storage unit.
  • the control unit 202 Controlling the first non-volatile storage unit 203 to dump the log record of the second data in the first non-volatile storage unit 203 in the second non-volatile storage unit 204, in the first non-volatile storage
  • the unit 203 successfully dumps the log record of the second data to the second non-volatile storage unit 204, the log of releasing the second data is recorded in the storage space occupied by the first non-volatile storage unit 203, so that the first The remaining space of the nonvolatile memory unit 203 is increased.
  • control unit 202 may further receive a read data request from the virtual machine, where the read data request includes a data identifier to be read out, and read the data to be read from the memory according to the to-be-read data identifier. Returning the data to be read back to the virtual machine.
  • the data processing device 40 includes a memory 401, an IO control module 402, an IO statistics module 403, a first non-volatile storage unit 404, and a second non- Volatile storage unit 405, the first non-volatile storage unit 404 writes data at a higher speed than the second non-volatile storage unit 405 writes data.
  • the control unit may include an IO control module 402 and an IO statistic module 403.
  • the data is divided into system control data and the upper layer. Read and write data requests.
  • the IO control module 402 has two main states: suspend and normal read and write. When the IO control module 402 is in the suspended state, the entire device does not respond to the read/write request; when the IO control module 402 is in the normal read/write state. The entire device can normally respond to the read and write request; specifically, the IO control module 402 performs data loading during the system startup, operation, and restart, and is responsible for controlling the read and write data requests, including querying the IO statistics module 403, setting the IO delay. Time, distribute IO to different storage media, control data dump operations, and more.
  • the IO statistics module 403 is responsible for counting information such as the number of times of reading and writing data requests, time, completion time, storage medium capacity, and the like, and collating and analyzing statistical information.
  • the memory 401 is a high-speed, large-capacity, volatile main memory, and needs to respond to all read and write data requests.
  • the first non-volatile storage unit 404 is a high-speed, small-capacity, non-volatile cache medium, and is responsible for buffering all write request data.
  • the second non-volatile storage unit 405 is a low-speed, large-capacity, non-volatile persistent medium, and the first non-volatile storage unit 404 And second non-volatile storage Unit 405 operates in conjunction to persist all of the write request data.
  • the read data request response is made, the response is made by the memory 401; when the write data request response is made, the first non-volatile storage unit 404 and the second non-volatile storage unit 405 respond, and at the first non- When the remaining space of the volatile storage unit 404 is insufficient, the write request data is dumped into the second nonvolatile storage unit 405 in an asynchronous manner.
  • the read and write data request and data dump process are managed by the IO control module 402 and the IO statistics module 403 to achieve high speed response to read and write data requests.
  • FIG. 5 is a flowchart of a method for data processing according to Embodiment 2 of the present invention. The method is applied to the data processing device 40 shown in FIG. 4. As shown in FIG. 5, the method specifically includes:
  • the virtual machine 400 sends a write data request to the IO control module 402, where the write data request includes the first data.
  • the virtual machine 400 When the user writes data to the computer, the virtual machine 400 issues a write data request to the IO control module 402, the write data request including the first data to be written.
  • the IO control module 402 receives the write data request from the virtual machine, determines a log record of the first data, and sends the log record size of the first data to the IO statistics module 403.
  • the IO statistics module 403 receives the size of the log record of the first data sent by the IO control module 402, compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit 404, and sends the IO control module to the IO control module. 402 returns the comparison result information.
  • S530 may be replaced by S531, which is implemented by: the IO control module 402 sends query request information to the IO statistics module 403, where the query request information is used to query the remaining space of the first non-volatile storage unit 404; The remaining space of the first nonvolatile storage unit 404 is acquired from the IO statistics module 403; the size of the log record of the first data is compared with the remaining space of the first nonvolatile storage unit 404.
  • the IO control module 402 determines, according to the comparison result information, whether the log record of the first data can be written to the first non-volatile storage unit 404. If yes, step S580 is performed; otherwise, step S550 is performed.
  • the IO control module 402 sends a data dump request letter to the first non-volatile storage unit 404.
  • the data dump request information includes address information of the second data.
  • the first non-volatile storage unit 404 dumps the log record of the second data to the second non-volatile storage unit 405 according to the address information of the second data, and receives the second non-volatile storage unit. After the information of the successfully dumped 405 is returned, the storage space occupied by the log record of the second data is released.
  • the IO statistics module 403 compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit 404, returns the comparison result information to the IO control module 402, and returns to S540. .
  • the S570 is replaced by S571.
  • the S571 is specifically implemented as: after delaying the specified threshold time, the IO control module 402 sends the query request information to the IO statistic module 403 again; and obtains the first non-easy information from the IO statistic module 403. The remaining space of the memory unit 404 is compared; the size of the log record of the first data is compared with the remaining space of the first nonvolatile storage unit 404, and the process returns to S540.
  • the IO control module 402 sends a write data request to the memory 401.
  • the IO statistics module 403 records the start time of the write data request response.
  • the memory 401 receives the write data request sent by the IO control module 402, writes a log record of the first data, and sends the write data request to the first non-volatile storage unit 404.
  • the first non-volatile storage unit 404 receives the write data request sent by the memory 401, writes the log record of the first data, and returns the successfully written information to the IO control module 402.
  • the first non-volatile storage unit 404 determines whether the write speed of the first non-volatile storage unit 404 is not greater than a preset threshold, and is not greater than When the threshold is set, the log record of the first data is written to the first nonvolatile storage unit 404. It should be noted that when the write speed is greater than the preset threshold, the system is in a high load state. If the write request processing is performed again, the read/write request processing error or the system crash may easily occur.
  • the preset threshold value is determined according to the number of times the device performs the read/write request processing. When the number of times the device performs the read/write request processing is relatively large, the preset threshold may be set longer, but generally All are in milliseconds. On the contrary, the preset threshold can be set shorter.
  • the IO control module 402 returns the successfully written information to the virtual machine, and sends the size of the log record of the first data to the IO statistics module 403.
  • the IO statistics module 403 updates the remaining space of the first non-volatile storage unit 404 according to the size of the log record of the first data.
  • the IO statistics module 403 calculates the response to the write data according to the recorded start time of the write data request response and the time of receiving the log record of the first data.
  • the remaining space of unit 404, the reduced value is the log record size of the first data.
  • the above IO control module 402 can control the delay of the write data request based on the IO statistics module 403, so as to ensure that the user data is not lost when responding to the write data request at high speed.
  • the data processing method when determining that the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 404, the second of the first non-volatile storage unit 404 The log record of the data is dumped in the second non-volatile storage unit 405, and when the remaining space of the first non-volatile storage unit 404 is sufficiently large, the data is written to the first non-volatile storage unit 404. Thereby the probability of failure to write data can be reduced.
  • FIG. 6 is a flowchart of a method for data dumping according to Embodiment 3 of the present invention. As shown in FIG. 6, the method specifically includes:
  • the IO statistics module 403 sends data dump indication information to the IO control module 402, where the data dump indication information The address information of the second data is included.
  • the IO statistics module 403 After the IO statistics module 403 sends the data dump indication information to the IO control module 402, the IO statistics module 302 records the start time of the data dump request response.
  • the IO control module 402 generates a data dump request signal according to the address information of the second data. And transmitting the data dump request information to the first non-volatile storage unit 404.
  • the first non-volatile storage unit 404 dumps the log record of the second data in the second non-volatile storage unit 405 according to the data dump request information.
  • the first non-volatile storage unit 404 releases the storage space occupied by the log record of the second data, so that the first The remaining space of a non-volatile storage unit 404 is increased and the information of the successful dump is returned to the IO control module 402.
  • the IO control module 402 sends the size of the log record of the second data to the IO statistics module 403.
  • the IO statistics module 403 updates the remaining space of the first nonvolatile storage unit 404 and the second nonvolatile storage unit 405 according to the size of the log record of the second data.
  • the IO statistic module 403 calculates the time taken to perform the data dump request processing according to the recorded start time of the data dump request response and the time of receiving the log record of the second data, and increases The remaining space of the recorded first nonvolatile storage unit 404 reduces the remaining space of the recorded second nonvolatile storage unit 405, and the increased and decreased values are the log record sizes of the second data.
  • the memory 401 is also stored, so the IO control module 402 can select to read the required data from the first non-volatile storage unit 404 according to the response speed of the first non-volatile storage unit 404 or the like, or directly from the memory.
  • the corresponding data is read in 401.
  • FIG. 7 is a schematic diagram of a data processing device according to Embodiment 4 of the present invention, where the device may be used to perform the method shown in FIG. 2 or FIG. 3.
  • the data processing device includes: a control unit 701, a first non-volatile storage unit 702, and a second non-volatile storage unit 703, the first The non-volatile memory unit 702 writes data at a higher speed than the second non-volatile storage unit 703 writes data.
  • the control unit 701 is configured to receive a write data request from the virtual machine, where the write data request includes the first data.
  • the first non-volatile storage unit 702 is configured to dump the log record of the second data to the second non-volatile storage unit 703.
  • the second non-volatile storage unit 703 is configured to write a log record of the second data.
  • control unit 701 includes an IO control module 704 and an IO statistics module 705;
  • the IO control module 704 is configured to send a log record size of the first data to the IO statistic module 705; the IO statistic module 705 saves a remaining space of the first non-volatile storage unit 702. The size of the log record of the first data sent by the IO control module 704 is compared with the remaining space of the first non-volatile storage unit 702, and the comparison result information is returned to the IO control module 704.
  • the IO statistic module 705 is further configured to: after the specified threshold time is delayed, if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 702, compare the The size of the log record of the first data is the remaining space of the first non-volatile storage unit 702.
  • control unit 701 includes an IO control module 704 and an IO statistics module 705;
  • the IO control module 704 is configured to send query request information to the IO statistic module 705, where the query request information is used to query the remaining space of the first non-volatile storage unit 702. Obtaining a remaining space of the first non-volatile storage unit 702 from the IO statistic module 705; comparing a size of the log record of the first data with a remaining of the first non-volatile storage unit 702 space.
  • the IO statistics module 705 sends the query request information.
  • the IO statistic module 705 is further configured to send data dump indication information to the IO control module 704, where the data dump indication information includes: address information of the second data; the IO control module The 704 is further configured to generate data dump request information according to the address information of the second data sent by the IO statistics module 705, and send the data dump request information to the first non-volatile storage unit 702.
  • the second non-volatile storage unit 703 is further configured to return information that is successfully dumped to the first non-volatile storage unit 702;
  • the first nonvolatile storage unit 702 is further configured to dump the log record of the second data to the second nonvolatile storage according to the data dump request information sent by the IO control module 704. After the information about the successful dump returned by the second non-volatile storage unit 703 is received, the storage space occupied by the log record of the second data is released, so that the first non-easy The remaining space of the lossy memory unit 702 is increased.
  • the IO control module 704 is further configured to send, according to the successfully dumped information, the size of the log record of the second data to the IO statistic module 705; the IO statistic module 705 is further configured to The size of the log record of the second data sent by the IO control module 704 updates the remaining space of the first non-volatile storage unit 702 and the second non-volatile storage unit 703.
  • the device further includes: a writing unit 706, configured to: if a size of the log record of the first data is not greater than a remaining space of the first non-volatile storage unit 702, and the first When the writing speed of the nonvolatile memory unit 702 is not greater than a preset threshold, the first A log of data is written to the first non-volatile storage unit 702.
  • a writing unit 706 configured to: if a size of the log record of the first data is not greater than a remaining space of the first non-volatile storage unit 702, and the first When the writing speed of the nonvolatile memory unit 702 is not greater than a preset threshold, the first A log of data is written to the first non-volatile storage unit 702.
  • the first non-volatile storage unit 702 is further configured to return information that is successfully written to the IO control module 704; the IO control module 704 is further configured to receive the first non-volatile The successfully written information returned by the storage unit 702 is sent to the IO statistics module 705 for the size of the log record of the first data;
  • the IO statistic module 705 is further configured to update the remaining space of the first non-volatile storage unit 702 according to the size of the log record of the first data sent by the IO control module 704.
  • control unit 701 is further configured to receive a read data request from the virtual machine, where the read data request includes a data identifier to be read out, and read the memory from the memory according to the to-be-read data identifier. Data to be read; the data to be read is returned to the virtual machine.
  • the device provided in the fourth embodiment of the present invention is embedded in the method provided in the second embodiment of the present invention. Therefore, the specific working process of the device provided by the present invention is not described herein.
  • the data processing method and device when determining that the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 702, the first non-volatile storage unit 702 The log record of the two data is dumped in the second non-volatile storage unit 703, and when the remaining space of the first non-volatile storage unit 702 is sufficiently large, the data is written to the first non-volatile storage unit 702. Thus, the probability of failure to write data can be reduced.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention relates to a data processing method and device, the method is applied to the data processing device which includes a control unit, a first nonvolatile storage unit and a second nonvolatile storage unit, and the rate of writing data of the first nonvolatile storage unit is higher than the rate of writing data of the second nonvolatile storage unit. The method includes: the control unit receiving a writing data request from a virtual machine, the writing data request including a first data; determining the log record of the first data; comparing the size of the log record of the first data with the remain space of the first nonvolatile storage unit, if the size of the log record of the first data is bigger than the remain space of the first nonvolatile storage unit, the control unit controlling the first nonvolatile storage unit to store the log record of the second data into the second nonvolatile storage unit to increase the remain space of the first nonvolatile storage unit. Therefore, the probability of failing to write data into the first nonvolatile storage unit is reduced.

Description

数据处理的方法及设备Data processing method and device 技术领域Technical field
本发明涉及云计算领域，尤其涉及一种数据处理的方法及设备。The present invention relates to the field of cloud computing, and in particular, to a data processing method and device.
背景技术Background technique
虚拟化技术是云计算领域的关键技术之一，其主要原理是将一台物理机的物理资源虚拟化为多个可以独立运行操作系统的虚拟机，每个虚拟机都可以独立实现物理机的功能，互不干扰，虚拟机使用过程与物理机一致，可以认为虚拟机是物理机的一种抽象化形式。Virtualization technology is one of the key technologies in the field of cloud computing. The main principle is to virtualize the physical resources of a physical machine into multiple virtual machines that can run the operating system independently. Each virtual machine can independently implement the physical machine. The function does not interfere with each other. The virtual machine usage process is consistent with the physical machine. The virtual machine can be considered as an abstract form of the physical machine.
图1是现有技术的物理机虚拟化的示意图。图1中，硬件平台100上的各种物理资源(包括中央处理器102、内存103、存储104(磁盘、固态硬盘(Solid State Driver，SSD)等等)、网卡105)被虚拟化管理系统110抽象为多个虚拟机(Virtual Machine,VM)130，在图中以130a、130b、和130c示例，其中物理存储104可以位于本地或远端(位于远端时需通过存储网络进行访问)。虚拟资源的功能与真实物理资源的作用完全一致，用户在虚拟机中对虚拟资源的操作与物理机完全一致。在不同的虚拟机中可以安装独立的操作系统132，以及一个或多个应用程序131。与图1不同的另外一种通用虚拟化示意中，虚拟硬件平台可以被包括在虚拟机130中。1 is a schematic diagram of a prior art physical machine virtualization. In FIG. 1, various physical resources on the hardware platform 100 (including the central processing unit 102, the memory 103, the storage 104 (disk, solid state drive (SSD), etc.), the network card 105) are virtualized by the management system 110. The abstraction is a plurality of virtual machines (VMs) 130, exemplified by 130a, 130b, and 130c, wherein the physical storage 104 can be local or remote (accessed through the storage network when located remotely). The function of the virtual resource is exactly the same as that of the real physical resource. The operation of the virtual resource in the virtual machine is completely consistent with the physical machine. A separate operating system 132, as well as one or more applications 131, can be installed in different virtual machines. In another general virtualization diagram different from FIG. 1, a virtual hardware platform may be included in the virtual machine 130.
虚拟机管理系统110包括多个软件组件，包括虚拟机监视器(Virtual Machine Monitor，VMM)120，在图中以120a、120b、和120c示例；管理虚拟化的软件组件等等。管理系统负责将物理存储104抽象为各个独立的虚拟存储124，供不同的虚拟机130使用，所有虚拟机130表面上使用的是各个独立的虚拟存储124，实际上则是使用的一个或多个物理存储104中的不同空间 (以文件或其他形式占用这部分空间)。当某个虚拟机130访问其对应的虚拟存储124时，管理系统110将对虚拟存储124的请求映射为对物理存储104不同空间的请求，从而实现不同虚拟机130拥有各自独立的虚拟存储124，在使用上相互独立。The virtual machine management system 110 includes a plurality of software components, including a Virtual Machine Monitor (VMM) 120, exemplified by 120a, 120b, and 120c in the figure; managing virtualized software components, and the like. The management system is responsible for abstracting the physical storage 104 into separate virtual storages 124 for use by different virtual machines 130. All virtual machines 130 use separate virtual storages 124 on their surface, in effect one or more of them. Different spaces in physical storage 104 (Use this file or other form to occupy this part of the space). When a virtual machine 130 accesses its corresponding virtual storage 124, the management system 110 maps the request for the virtual storage 124 to a request for a different space of the physical storage 104, thereby enabling the different virtual machines 130 to have their own independent virtual storages 124, Independent of each other in use.
该方案抽象来看，写请求数据首先写入到高速、大容量、易失主存介质103(内存)，并同步写入到高速、小容量、非易失性随机存储器104(NVRAM)后返回。由于NVRAM容量通常比较小，从而可能出现写入失败的情况。In the abstraction of the scheme, the write request data is first written to the high-speed, large-capacity, volatile main storage medium 103 (memory), and is synchronously written to the high-speed, small-capacity, non-volatile random access memory 104 (NVRAM) and returned. Since the NVRAM capacity is usually small, write failure may occur.
发明内容Summary of the invention
本发明实施例提供了一种数据处理的方法及设备，能减小向NVRAM写入数据失败的概率。Embodiments of the present invention provide a data processing method and device, which can reduce the probability of failure to write data to NVRAM.
第一方面，提供了一种数据处理的方法，该方法应用于数据处理设备，所述数据处理设备包括控制单元，第一非易失性存储单元和第二非易失性存储单元，所述第一非易失性存储单元写数据的速度高于所述第二非易失性存储单元写数据的速度，所述方法包括：In a first aspect, a method of data processing is provided, the method being applied to a data processing device, the data processing device comprising a control unit, a first non-volatile storage unit and a second non-volatile storage unit, The speed at which the first non-volatile memory unit writes data is higher than the speed at which the second non-volatile memory unit writes data, the method comprising:
所述控制单元从虚拟机接收写数据请求，所述写数据请求包括第一数据；The control unit receives a write data request from a virtual machine, the write data request including first data;
确定所述第一数据的日志记录；Determining a log record of the first data;
比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则所述控制单元控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据的日志记录转储于所述第二非易失性存储单元，以使所述第一非易失性存储单元的剩余空间增加。Comparing a size of the log record of the first data with a remaining space of the first non-volatile storage unit, if a size of the log record of the first data is greater than a remaining of the first non-volatile storage unit Space, the control unit controls the first non-volatile storage unit to dump a log record of the second data in the first non-volatile storage unit to the second non-volatile storage unit So that the remaining space of the first non-volatile memory unit is increased.
根据第一方面，在第一方面的第二种可能的实现方式中，所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间之前，所述方法还包括： According to the first aspect, in a second possible implementation manner of the first aspect, before the comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit, the method further Includes:
所述控制单元的IO控制模块将所述第一数据的日志记录大小发送给所述控制单元的IO统计模块，所述IO统计模块保存所述第一非易失性存储单元的剩余空间；The IO control module of the control unit sends a log record size of the first data to an IO statistic module of the control unit, where the IO statistic module saves a remaining space of the first nonvolatile storage unit;
所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间包括：The comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
所述IO统计模块比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，并向所述IO控制模块返回比较结果信息。The IO statistic module compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit, and returns comparison result information to the IO control module.
根据第一方面的第一种可能的实现方式，在第一方面的第二种可能的实现方式中，所述方法还包括：According to a first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the method further includes:
若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，所述IO统计模块再次比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。If the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, after the specified threshold time is delayed, the IO statistics module compares the size of the log record of the first data again. And a remaining space of the first nonvolatile storage unit.
根据第一方面，在第一方面的第三种可能的实现方式中，所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间之前，所述方法还包括：According to a first aspect, in a third possible implementation manner of the first aspect, before the comparing the size of the log record of the first data with the remaining space of the first nonvolatile storage unit, the method further include:
所述控制单元的IO控制模块向所述控制单元的IO统计模块发送查询请求信息，所述查询请求信息用于查询所述第一非易失性存储单元的剩余空间；The IO control module of the control unit sends query request information to the IO statistics module of the control unit, where the query request information is used to query the remaining space of the first non-volatile storage unit;
所述IO控制模块从所述IO统计模块获取所述第一非易失性存储单元的剩余空间；The IO control module acquires a remaining space of the first non-volatile storage unit from the IO statistic module;
所述比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间包括：The comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
所述IO控制模块比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。The IO control module compares a size of the log record of the first data with a remaining space of the first non-volatile storage unit.
根据第一方面的第三种可能的实现方式，在第一方面的第四种可能的实现方式中，所述方法还包括： According to a third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the method further includes:
若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，所述IO控制模块再次向所述IO统计模块发送所述查询请求信息。If the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, after the specified threshold time is delayed, the IO control module sends the query request to the IO statistic module again. information.
根据第一方面的第一种可能的实现方式或者第一方面的第二种可能的实现方式，在第一方面的第五种可能的实现方式中，所述若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，所述控制单元控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据转储于所述第二非易失性存储单元包括：According to the first possible implementation of the first aspect, or the second possible implementation of the first aspect, in a fifth possible implementation manner of the first aspect, the logging of the first data The size of the first non-volatile storage unit is larger than the remaining space of the first non-volatile storage unit, and the control unit controls the first non-volatile storage unit to dump the second data in the first non-volatile storage unit The second non-volatile storage unit includes:
所述IO统计模块向所述IO控制模块发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息；The IO statistic module sends data dump indication information to the IO control module, where the data dump indication information includes: address information of the second data;
所述IO控制模块根据所述第二数据的地址信息生成数据转储请求信息，向所述第一非易失性存储单元发送所述数据转储请求信息，以使所述第一非易失性存储单元根据所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元，并且在接收到所述第二非易失性存储单元返回的成功转储的信息后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元的剩余空间增加。The IO control module generates data dump request information according to the address information of the second data, and sends the data dump request information to the first non-volatile storage unit to enable the first non-volatile The storage unit dumps the log record of the second data to the second non-volatile storage unit according to the data dump request information, and receives the return of the second non-volatile storage unit After successfully dumping the information, the storage space occupied by the log record of the second data is released to increase the remaining space of the first non-volatile storage unit.
根据第一方面的第五种可能的实现方式，在第一方面的第六种可能的实现方式中，所述在接收到所述第二非易失性存储单元返回的成功转储的信息后，所述方法还包括：According to a fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, after the information about the successful dump returned by the second non-volatile storage unit is received The method further includes:
所述IO控制模块向所述IO统计模块发送所述第二数据的日志记录的大小，由所述IO统计模块根据所述第二数据的日志记录的大小更新所述第一非易失性存储单元和所述第二非易失性存储单元的剩余空间。The IO control module sends a size of the log record of the second data to the IO statistic module, and the IO statistic module updates the first non-volatile storage according to the size of the log record of the second data. a remaining space of the unit and the second non-volatile storage unit.
根据第一方面，在第一方面的第七种可能的实现方式中，所述方法还包括：According to the first aspect, in a seventh possible implementation manner of the first aspect, the method further includes:
若所述第一数据的日志记录的大小不大于所述第一非易失性存储单元的剩余空间，并且所述第一非易失性存储单元的写入速度不大于预设的阈 值时，将所述第一数据的日志记录写入所述第一非易失性存储单元。If the size of the log record of the first data is not greater than the remaining space of the first non-volatile storage unit, and the write speed of the first non-volatile storage unit is not greater than a preset threshold At the time of the value, the log record of the first data is written to the first non-volatile storage unit.
根据第一方面的第七种可能的实现方式，在第一方面的第八种可能的实现方式中，所述在将所述第一数据的日志记录写入所述第一非易失性存储单元之后，所述方法还包括：According to a seventh possible implementation manner of the first aspect, in an eighth possible implementation manner of the first aspect, the logging of the first data is written to the first non-volatile storage After the unit, the method further includes:
所述IO控制模块接收所述第一非易失性存储单元返回的成功写入的信息，向所述IO统计模块发送所述第一数据的日志记录的大小，由所述IO统计模块根据所述第一数据的日志记录的大小更新所述第一非易失性存储单元的剩余空间。The IO control module receives the successfully written information returned by the first non-volatile storage unit, and sends the size of the log record of the first data to the IO statistics module, according to the IO statistics module. The size of the log record of the first data updates the remaining space of the first non-volatile storage unit.
根据第一方面或者第一方面的上述八种可能的实现方式中任一种可能的实现方式，在第一方面的第九种可能的实现方式中，所述方法还包括：In a ninth possible implementation manner of the first aspect, the method further includes:
所述控制单元从所述虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；The control unit receives a read data request from the virtual machine, and the read data request includes a data identifier to be read out;
根据所述待读出数据标识，从所述内存读取所述待读出数据；Reading the data to be read from the memory according to the data identifier to be read;
将所述待读出数据返回给所述虚拟机。Returning the data to be read to the virtual machine.
第二方面，提供了一种数据处理设备，该数据处理设备包括：控制单元，第一非易失性存储单元和第二非易失性存储单元，所述第一非易失性存储单元写数据的速度高于所述第二非易失性存储单元写数据的速度；In a second aspect, a data processing apparatus is provided, the data processing apparatus comprising: a control unit, a first non-volatile storage unit and a second non-volatile storage unit, the first non-volatile storage unit writing The speed of the data is higher than the speed at which the second non-volatile storage unit writes data;
所述控制单元，用于从虚拟机接收写数据请求，所述写数据请求包括第一数据；The control unit is configured to receive a write data request from a virtual machine, where the write data request includes first data;
确定所述第一数据的日志记录；Determining a log record of the first data;
比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据的日志记录转储于所述第二非易失性存储单元，以使所述第一非易失性存储单元的剩余空间增加；Comparing a size of the log record of the first data with a remaining space of the first non-volatile storage unit, if a size of the log record of the first data is greater than a remaining of the first non-volatile storage unit Space, controlling the first non-volatile storage unit to dump a log record of the second data in the first non-volatile storage unit to the second non-volatile storage unit to cause the The remaining space of the first non-volatile memory unit is increased;
所述第一非易失性存储单元，用于将所述第二数据的日志记录转储于 所述第二非易失性存储单元；The first non-volatile storage unit is configured to dump the log record of the second data to The second nonvolatile memory unit;
所述第二非易失性存储单元，用于写入所述第二数据的日志记录。The second non-volatile storage unit is configured to write a log record of the second data.
所述控制单元包括IO控制模块和IO统计模块；The control unit includes an IO control module and an IO statistics module;
所述IO控制模块，用于将所述第一数据的日志记录大小发送给所述IO统计模块；The IO control module is configured to send a log record size of the first data to the IO statistic module;
所述IO统计模块保存所述第一非易失性存储单元的剩余空间，用于比较所述IO控制模块发送的第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，并向所述IO控制模块返回比较结果信息。The IO statistics module saves a remaining space of the first non-volatile storage unit, and compares a size of a log record of the first data sent by the IO control module with the first non-volatile storage unit. Remaining space and returning comparison result information to the IO control module.
根据第二方面，在第二方面的第二种可能的实现方式中，所述IO统计模块还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，再次比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。According to the second aspect, in a second possible implementation manner of the second aspect, the IO statistic module is further configured to: if a size of the log record of the first data is greater than that of the first non-volatile storage unit The remaining space, after delaying the specified threshold time, compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit again.
根据第二方面的第一种可能的实现方式，在第二方面的第二种可能的实现方式中，所述控制单元包括IO控制模块和IO统计模块；According to a first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the control unit includes an IO control module and an IO statistics module;
所述IO控制模块，用于向所述IO统计模块发送查询请求信息，所述查询请求信息用于查询所述第一非易失性存储单元的剩余空间；The IO control module is configured to send query request information to the IO statistic module, where the query request information is used to query a remaining space of the first non-volatile storage unit;
从所述IO统计模块获取所述第一非易失性存储单元的剩余空间；Obtaining a remaining space of the first nonvolatile storage unit from the IO statistic module;
比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。Comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit.
根据第二方面，在第二方面的第三种可能的实现方式中，所述IO控制模块还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，再次向所述IO统计模块发送所述查询请求信息。According to a second aspect, in a third possible implementation of the second aspect, the IO control module is further configured to: if a size of the log record of the first data is greater than that of the first non-volatile storage unit After the remaining space is delayed by the specified threshold time, the query request information is sent to the IO statistic module again.
根据第二方面的第三种可能的实现方式，在第二方面的第四种可能的实现方式中，所述IO统计模块还用于向所述IO控制模块发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息； According to a third possible implementation manner of the second aspect, in a fourth possible implementation manner of the second aspect, the IO statistic module is further configured to send data dump indication information to the IO control module, The data dump indication information includes: address information of the second data;
所述IO控制模块，还用于根据所述IO统计模块发送的第二数据的地址信息生成数据转储请求信息，向所述第一非易失性存储单元发送所述数据转储请求信息；The IO control module is further configured to generate data dump request information according to the address information of the second data sent by the IO statistics module, and send the data dump request information to the first non-volatile storage unit;
所述第二非易失性存储单元，还用于向所述第一非易失存储单元返回成功转储的信息；The second non-volatile storage unit is further configured to return information that is successfully dumped to the first non-volatile storage unit;
所述第一非易失存储单元，还用于根据所述IO控制模块发送的所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元；在接收到所述第二非易失性存储单元返回的所述成功转储的信息后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元的剩余空间增加。The first nonvolatile storage unit is further configured to dump the log record of the second data to the second nonvolatile storage unit according to the data dump request information sent by the IO control module After receiving the successfully dumped information returned by the second non-volatile storage unit, releasing the storage space occupied by the log record of the second data, so that the first non-volatile storage The remaining space of the unit increases.
根据第二方面的第一种可能的实现方式或者第二方面的第二种可能的实现方式，在第二方面的第五种可能的实现方式中，所述IO控制模块还用于根据所述成功转储的信息向所述IO统计模块发送所述第二数据的日志记录的大小；In a fifth possible implementation manner of the second aspect, the IO control module is further configured to be used according to the first possible implementation manner of the second aspect, or the second possible implementation manner of the second aspect The successfully dumped information sends the size of the log record of the second data to the IO statistics module;
所述IO统计模块，还用于根据所述IO控制模块发送的所述第二数据的日志记录的大小更新所述第一非易失性存储单元和所述第二非易失性存储单元的剩余空间。The IO statistic module is further configured to update, according to the size of the log record of the second data sent by the IO control module, the first non-volatile storage unit and the second non-volatile storage unit remaining space.
根据第二方面的第五种可能的实现方式，在第二方面的第六种可能的实现方式中，所述设备还包括：写入单元，用于若所述第一数据的日志记录的大小不大于所述第一非易失性存储单元的剩余空间，并且所述第一非易失性存储单元的写入速度不大于预设的阈值时，将所述第一数据的日志记录写入所述第一非易失性存储单元。According to a fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, the device further includes: a writing unit, configured, if the size of the first data is recorded Writing not less than the remaining space of the first nonvolatile storage unit, and writing the log record of the first data when the write speed of the first nonvolatile storage unit is not greater than a preset threshold The first non-volatile storage unit.
根据第二方面，在第二方面的第七种可能的实现方式中，所述第一非易失性存储单元，还用于向所述IO控制模块返回成功写入的信息；According to a second aspect, in a seventh possible implementation of the second aspect, the first non-volatile storage unit is further configured to return information that is successfully written to the IO control module;
所述IO控制模块，还用于接收所述第一非易失性存储单元返回的成功写入的信息，向所述IO统计模块发送所述第一数据的日志记录的大小； The IO control module is further configured to receive information that is successfully written by the first non-volatile storage unit, and send a size of a log record of the first data to the IO statistics module;
所述IO统计模块，还用于根据所述IO控制模块发送的所述第一数据的日志记录的大小更新所述第一非易失性存储单元的剩余空间。The IO statistic module is further configured to update a remaining space of the first non-volatile storage unit according to a size of a log record of the first data sent by the IO control module.
根据第二方面或者第二方面的上述八种可能的实现方式中任一种可能的实现方式，在第二方面的第九种可能的实现方式中，所述控制单元还用于从所述虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；In a ninth possible implementation manner of the second aspect, in the ninth possible implementation manner of the second aspect, the control unit is further configured to use the virtual Receiving a read data request, the read data request including a data identifier to be read out;
根据所述待读出数据标识，从内存读取所述待读出数据；Reading the data to be read from the memory according to the data identifier to be read;
将所述待读出数据返回给所述虚拟机。Returning the data to be read to the virtual machine.
本发明实施例提供的数据处理的方法及设备，控制单元从虚拟机接收写数据请求，写数据请求包括第一数据；确定第一数据的日志记录；比较第一数据的日志记录的大小与第一非易失性存储单元的剩余空间，若第一数据的日志记录的大小大于第一非易失性存储单元的剩余空间，则控制单元控制第一非易失性存储单元将第二数据的日志记录转储于第二非易失性存储单元，以使第一非易失性存储单元的剩余空间增加。由此能减小向第一非易失性存储单元写入数据失败的概率。The data processing method and device provided by the embodiment of the present invention, the control unit receives a write data request from the virtual machine, the write data request includes the first data, determines the log record of the first data, and compares the size and the log record of the first data. a remaining space of a non-volatile storage unit, if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, the control unit controls the first non-volatile storage unit to The log record is dumped to the second non-volatile storage unit to increase the remaining space of the first non-volatile storage unit. Thereby, the probability of failure to write data to the first nonvolatile memory cell can be reduced.
附图说明DRAWINGS
图1是现有技术的物理机虚拟化的示意图；1 is a schematic diagram of a prior art physical machine virtualization;
图2为本发明实施例一提供的数据处理设备结构示意图；2 is a schematic structural diagram of a data processing device according to Embodiment 1 of the present invention;
图3为本发明实施例一提供的数据处理的方法流程图；3 is a flowchart of a method for data processing according to Embodiment 1 of the present invention;
图4为本发明实施例二提供的数据处理设备结构示意图；4 is a schematic structural diagram of a data processing device according to Embodiment 2 of the present invention;
图5为本发明实施例二提供的数据处理的方法流程图；FIG. 5 is a flowchart of a method for data processing according to Embodiment 2 of the present invention;
图6为本发明实施例三提供的数据转储的方法流程图；6 is a flowchart of a method for data dumping according to Embodiment 3 of the present invention;
图7为本发明实施例四提供的数据处理设备的示意图。FIG. 7 is a schematic diagram of a data processing device according to Embodiment 4 of the present invention.
具体实施方式detailed description
下面通过附图和实施例，对本发明的技术方案做进一步的详细描述。 The technical solution of the present invention will be further described in detail below through the accompanying drawings and embodiments.
在当前的虚拟机部署环境中，为节省资源，多个虚拟存储是位于同一个物理存储介质中的。这就造成，当多个虚拟机对各自的虚拟存储发送读写请求时，本质上是对同一个物理存储介质发起读写请求。物理存储介质有多项技术指标，其中的每秒进行读写操作的次数(Input/Output Operations Per Second，IOPS)指标决定了其响应读写请求的速度，其值越大，则响应越快。但在当前阶段，任何物理存储介质所能提供的IOPS数值都是有限的。具体的IOPS数值又与读写请求类型相关，读写请求类型可分为顺序读写、随机读写。顺序读写是指读写请求在存储介质的物理空间上是连续的，而随机读写是指读写请求在存储介质的物理空间上是非连续的、是间断的。一般说来，物理存储介质的顺序读写IOPS要远远大于随机读写IOPS，而在虚拟化环境中，多个虚拟机的读写请求在发给同一个物理存储时，会产生极大的随机性。最终各个虚拟存储的读写性能由于相互影响而迅速降低，极大影响虚拟机用户的使用体验。In the current virtual machine deployment environment, in order to save resources, multiple virtual storages are located on the same physical storage medium. This causes that when multiple virtual machines send read and write requests to their respective virtual storages, they essentially initiate read and write requests to the same physical storage medium. The physical storage medium has a number of technical indicators, and the number of times of reading/receiving operations per second (IOPS) determines the speed at which it responds to read and write requests. The larger the value, the faster the response. However, at this stage, the IOPS values that any physical storage medium can provide are limited. The specific IOPS value is related to the type of read and write requests. The read and write request types can be divided into sequential read and write and random read and write. Sequential read and write means that the read and write requests are continuous in the physical space of the storage medium, and random read and write means that the read and write requests are discontinuous and discontinuous in the physical space of the storage medium. In general, the sequential read and write IOPS of physical storage media is much larger than random read and write IOPS. In a virtualized environment, when multiple virtual machine read and write requests are sent to the same physical storage, it will generate a great deal. Randomness. Eventually, the read and write performance of each virtual storage is rapidly reduced due to mutual influence, which greatly affects the experience of the virtual machine user.
在虚拟化环境下使用的大容量存储介质主要包括内存、磁盘、固态硬盘(Solid State Driver，SSD)、非易失性随机存储器(Non Volatile Random Access Memory，NVRAM)等等。从速度上对比，内存、NVRAM等是IO响应速度最快的介质，其中内存在掉电后数据会丢失，NVRAM则不会，但NVRAM价格昂贵；从容量及成本上对比，磁盘是容量最大且成本最低的介质，但其IOPS最低，尤其是随机IOPS。这几种介质在速度、容量及成本上存在矛盾，虚拟化存储部署中一般会根据用户需求进行各种存储介质的搭配，使之在速度、容量及成本上达到一个平衡。The large-capacity storage media used in the virtualized environment mainly include a memory, a disk, a solid state drive (SSD), a non-volatile random access memory (NVRAM), and the like. From the speed comparison, memory, NVRAM, etc. is the fastest IO response medium, in which the data will be lost after the power is lost, NVRAM will not, but NVRAM is expensive; in terms of capacity and cost, the disk is the largest and The lowest cost media, but with the lowest IOPS, especially random IOPS. These kinds of media have contradictions in speed, capacity and cost. In virtualized storage deployment, various storage media are generally matched according to user requirements, so that a balance is achieved in speed, capacity and cost.
图2为本发明实施例一提供的数据处理设备20结构示意图，所述数据处理设备20包括内存201，控制单元202，第一非易失性存储单元203和第二非易失性存储单元204，所述第一非易失性存储单元203写数据的速度高于所述第二非易失性存储单元204写数据的速度，因此，能够实现业务数据的快速备份，尤其是对于内存中存储处理设备需要存储的业务数据的情况下，能够保证内 存数据的安全性。图2中，内存201为高速、大容量、易失主存介质，需要响应读写数据请求；第一非易失性存储单元203为高速、小容量、非易失缓存介质，负责缓存写请求数据，并与控制单元202结合，进行数据的转储；第二非易失性存储单元204为低速、大容量、非易失持久化介质，第一非易失性存储单元203和第二非易失性存储单元204联合操作，持久化存储写请求数据。当进行读数据请求响应时，由内存201进行响应；当进行写数据请求响应时，由第一非易失性存储单元203和第二非易失性存储单元204进行响应，并在第一非易失性存储单元203剩余空间不足时，采用异步方式将写请求数据转储到第二非易失性存储单元204中。读写数据请求及数据转储过程由控制单元202进行管理，以达到高速的响应读写数据请求的目的。FIG. 2 is a schematic structural diagram of a data processing device 20 according to Embodiment 1 of the present invention. The data processing device 20 includes a memory 201, a control unit 202, a first non-volatile storage unit 203, and a second non-volatile storage unit 204. The first non-volatile storage unit 203 writes data faster than the second non-volatile storage unit 204 writes data, thereby enabling fast backup of service data, especially for in-memory storage. When the processing device needs to store the service data, it can be guaranteed Store data security. In FIG. 2, the memory 201 is a high-speed, large-capacity, volatile main memory, and needs to respond to read and write data requests. The first non-volatile storage unit 203 is a high-speed, small-capacity, non-volatile cache medium, and is responsible for buffering write request data. And combining with the control unit 202 to perform data dumping; the second non-volatile storage unit 204 is a low-speed, large-capacity, non-volatile persistent medium, the first non-volatile storage unit 203 and the second non-easy The dysfunctional storage unit 204 operates in conjunction to persist store write request data. When the read data request response is made, the response is performed by the memory 201; when the write data request response is made, the first nonvolatile storage unit 203 and the second nonvolatile storage unit 204 respond, and at the first non- When the remaining space of the volatile storage unit 203 is insufficient, the write request data is dumped into the second nonvolatile storage unit 204 in an asynchronous manner. The read and write data request and data dump process are managed by the control unit 202 to achieve a high speed response to read and write data requests.
图3为本发明实施例一提供的数据处理的方法流程图，该方法应用于图2所示的数据处理设备20。如图3所示，所述方法具体包括：FIG. 3 is a flowchart of a method for data processing according to Embodiment 1 of the present invention, and the method is applied to the data processing device 20 shown in FIG. 2. As shown in FIG. 3, the method specifically includes:
S310，所述控制单元从虚拟机接收写数据请求，所述写数据请求包括第一数据。S310. The control unit receives a write data request from a virtual machine, where the write data request includes first data.
当用户向计算机写入数据时，虚拟机便向控制单元202发出写数据请求，该写数据请求中包括待写入的第一数据。When the user writes data to the computer, the virtual machine issues a write data request to the control unit 202, the write data request including the first data to be written.
S320，确定所述第一数据的日志记录。S320. Determine a log record of the first data.
需要说明的是，根据写数据请求中包括的第一数据确定第一数据的日志记录属于现有技术，在此不作赘述。It should be noted that determining the log record of the first data according to the first data included in the write data request belongs to the prior art, and details are not described herein.
S330，比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则所述控制单元控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据的日志记录转储于所述第二非易失性存储单元，以使所述第一非易失性存储单元的剩余空间增加。S330, comparing a size of the log record of the first data with a remaining space of the first non-volatile storage unit, if a size of the log record of the first data is greater than the first non-volatile storage unit The remaining space, the control unit controls the first non-volatile storage unit to dump the log record of the second data in the first non-volatile storage unit to the second non-volatile The storage unit is configured to increase the remaining space of the first nonvolatile storage unit.
举例来说，控制单元202接收到写数据请求之后，并且判断第一数据的日志记录的大小大于第一非易失性存储单元203的剩余空间时，控制单元202便 控制第一非易失性存储单元203将第一非易失性存储单元203中的第二数据的日志记录转储于第二非易失性存储单元204中，在第一非易失性存储单元203将第二数据的日志记录成功转储于第二非易失性存储单元204时，释放第二数据的日志记录在第一非易失性存储单元203占用的存储空间，以使第一非易失性存储单元203的剩余空间增加。For example, after the control unit 202 receives the write data request, and determines that the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 203, the control unit 202 Controlling the first non-volatile storage unit 203 to dump the log record of the second data in the first non-volatile storage unit 203 in the second non-volatile storage unit 204, in the first non-volatile storage When the unit 203 successfully dumps the log record of the second data to the second non-volatile storage unit 204, the log of releasing the second data is recorded in the storage space occupied by the first non-volatile storage unit 203, so that the first The remaining space of the nonvolatile memory unit 203 is increased.
可选地，控制单元202还可以从虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；根据所述待读出数据标识，从所述内存读取所述待读出数据；将所述待读出数据返回给所述虚拟机。Optionally, the control unit 202 may further receive a read data request from the virtual machine, where the read data request includes a data identifier to be read out, and read the data to be read from the memory according to the to-be-read data identifier. Returning the data to be read back to the virtual machine.
图4为本发明实施例二提供的数据处理设备40结构示意图，所述数据处理设备40包括内存401，IO控制模块402，IO统计模块403，第一非易失性存储单元404和第二非易失性存储单元405，所述第一非易失性存储单元404写数据的速度高于所述第二非易失性存储单元405写数据的速度。具体地，可以是控制单元包括IO控制模块402和IO统计模块403。当设备启动、运行及重启时，各模块在不同操作流程中的关联关系如图4中连线所示，连线的箭头方向代表了数据流向，其中的数据分为系统控制数据及上层下发的读写数据请求。具体地，IO控制模块402主要有两个状态：挂起和正常读写，当IO控制模块402为挂起状态时，整个设备不响应读写请求；当IO控制模块402为正常读写状态时，整个设备才能正常响应读写请求；具体地，IO控制模块402在系统启动、运行及重启时进行数据加载以及负责进行读写数据请求的控制，包括查询IO统计模块403、设定IO延时时间、分发IO给不同的存储介质、控制数据转储操作等等。IO统计模块403负责统计读写数据请求的次数、时间、完成耗时、存储介质容量等信息，并进行统计信息的整理、分析。内存401为高速、大容量、易失主存介质，需要响应所有读写数据请求；第一非易失性存储单元404为高速、小容量、非易失缓存介质，负责缓存所有的写请求数据，并与IO控制模块402和IO统计模块403结合，进行数据的转储；第二非易失性存储单元405为低速、大容量、非易失持久化介质，第一非易失性存储单元404和第二非易失性存储 单元405联合操作，持久化存储所有的写请求数据。当进行读数据请求响应时，由内存401进行响应；当进行写数据请求响应时，由第一非易失性存储单元404和第二非易失性存储单元405进行响应，并在第一非易失性存储单元404剩余空间不足时，采用异步方式将写请求数据转储到第二非易失性存储单元405中。读写数据请求及数据转储过程由IO控制模块402和IO统计模块403进行管理，以达到高速的响应读写数据请求的目的。4 is a schematic structural diagram of a data processing device 40 according to Embodiment 2 of the present invention. The data processing device 40 includes a memory 401, an IO control module 402, an IO statistics module 403, a first non-volatile storage unit 404, and a second non- Volatile storage unit 405, the first non-volatile storage unit 404 writes data at a higher speed than the second non-volatile storage unit 405 writes data. Specifically, the control unit may include an IO control module 402 and an IO statistic module 403. When the device is started, running, and restarted, the relationship between the modules in different operation processes is as shown in the connection in Figure 4. The direction of the arrow of the connection represents the data flow direction. The data is divided into system control data and the upper layer. Read and write data requests. Specifically, the IO control module 402 has two main states: suspend and normal read and write. When the IO control module 402 is in the suspended state, the entire device does not respond to the read/write request; when the IO control module 402 is in the normal read/write state. The entire device can normally respond to the read and write request; specifically, the IO control module 402 performs data loading during the system startup, operation, and restart, and is responsible for controlling the read and write data requests, including querying the IO statistics module 403, setting the IO delay. Time, distribute IO to different storage media, control data dump operations, and more. The IO statistics module 403 is responsible for counting information such as the number of times of reading and writing data requests, time, completion time, storage medium capacity, and the like, and collating and analyzing statistical information. The memory 401 is a high-speed, large-capacity, volatile main memory, and needs to respond to all read and write data requests. The first non-volatile storage unit 404 is a high-speed, small-capacity, non-volatile cache medium, and is responsible for buffering all write request data. And combining with the IO control module 402 and the IO statistics module 403 to perform data dumping; the second non-volatile storage unit 405 is a low-speed, large-capacity, non-volatile persistent medium, and the first non-volatile storage unit 404 And second non-volatile storage Unit 405 operates in conjunction to persist all of the write request data. When the read data request response is made, the response is made by the memory 401; when the write data request response is made, the first non-volatile storage unit 404 and the second non-volatile storage unit 405 respond, and at the first non- When the remaining space of the volatile storage unit 404 is insufficient, the write request data is dumped into the second nonvolatile storage unit 405 in an asynchronous manner. The read and write data request and data dump process are managed by the IO control module 402 and the IO statistics module 403 to achieve high speed response to read and write data requests.
图5为本发明实施例二提供的数据处理的方法流程图，该方法应用于图4所示的数据处理设备40。如图5所示，所述方法具体包括：FIG. 5 is a flowchart of a method for data processing according to Embodiment 2 of the present invention. The method is applied to the data processing device 40 shown in FIG. 4. As shown in FIG. 5, the method specifically includes:
S510，虚拟机400向IO控制模块402发送写数据请求，该写数据请求包括第一数据。S510. The virtual machine 400 sends a write data request to the IO control module 402, where the write data request includes the first data.
当用户向计算机写入数据时，虚拟机400便向IO控制模块402发出写数据请求，该写数据请求中包括待写入的第一数据。When the user writes data to the computer, the virtual machine 400 issues a write data request to the IO control module 402, the write data request including the first data to be written.
S520，IO控制模块402从虚拟机接收写数据请求，确定第一数据的日志记录，并将第一数据的日志记录大小发送给IO统计模块403。S520. The IO control module 402 receives the write data request from the virtual machine, determines a log record of the first data, and sends the log record size of the first data to the IO statistics module 403.
S530，IO统计模块403接收IO控制模块402发送的第一数据的日志记录的大小，比较第一数据的日志记录的大小与第一非易失性存储单元404的剩余空间，并向IO控制模块402返回比较结果信息。S530, the IO statistics module 403 receives the size of the log record of the first data sent by the IO control module 402, compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit 404, and sends the IO control module to the IO control module. 402 returns the comparison result information.
可选地，S530可以替换为S531，S531具体实现为：IO控制模块402向IO统计模块403发送查询请求信息，所述查询请求信息用于查询第一非易失性存储单元404的剩余空间；从IO统计模块403获取第一非易失性存储单元404的剩余空间；比较第一数据的日志记录的大小与第一非易失性存储单元404的剩余空间。Alternatively, S530 may be replaced by S531, which is implemented by: the IO control module 402 sends query request information to the IO statistics module 403, where the query request information is used to query the remaining space of the first non-volatile storage unit 404; The remaining space of the first nonvolatile storage unit 404 is acquired from the IO statistics module 403; the size of the log record of the first data is compared with the remaining space of the first nonvolatile storage unit 404.
S540，IO控制模块402根据比较结果信息判断是否可以向第一非易失性存储单元404写入第一数据的日志记录，是则，执行步骤S580；否则执行步骤S550。S540. The IO control module 402 determines, according to the comparison result information, whether the log record of the first data can be written to the first non-volatile storage unit 404. If yes, step S580 is performed; otherwise, step S550 is performed.
S550，IO控制模块402向第一非易失性存储单元404发送数据转储请求信 息，该数据转储请求信息包括第二数据的地址信息。S550, the IO control module 402 sends a data dump request letter to the first non-volatile storage unit 404. The data dump request information includes address information of the second data.
S560，第一非易失性存储单元404根据第二数据的地址信息将第二数据的日志记录转储于第二非易失性存储单元405，并且在接收到第二非易失性存储单元405返回的成功转储的信息后，释放所述第二数据的日志记录占用的存储空间。S560, the first non-volatile storage unit 404 dumps the log record of the second data to the second non-volatile storage unit 405 according to the address information of the second data, and receives the second non-volatile storage unit. After the information of the successfully dumped 405 is returned, the storage space occupied by the log record of the second data is released.
S570，延迟指定阈值时间之后，IO统计模块403再次比较所述第一数据的日志记录的大小与第一非易失性存储单元404的剩余空间，向IO控制模块402返回比较结果信息，返回S540。S570. After delaying the specified threshold time, the IO statistics module 403 compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit 404, returns the comparison result information to the IO control module 402, and returns to S540. .
当步骤S530替换为S531时，S570则替换为S571，S571具体实现为：延迟指定阈值时间之后，IO控制模块402再次向IO统计模块403发送查询请求信息；从IO统计模块403获取第一非易失性存储单元404的剩余空间；比较第一数据的日志记录的大小与第一非易失性存储单元404的剩余空间，返回S540。When the step S530 is replaced by S531, the S570 is replaced by S571. The S571 is specifically implemented as: after delaying the specified threshold time, the IO control module 402 sends the query request information to the IO statistic module 403 again; and obtains the first non-easy information from the IO statistic module 403. The remaining space of the memory unit 404 is compared; the size of the log record of the first data is compared with the remaining space of the first nonvolatile storage unit 404, and the process returns to S540.
S580，IO控制模块402向内存401发送写数据请求。S580, the IO control module 402 sends a write data request to the memory 401.
IO控制模块402向内存401发送写数据请求之后，由IO统计模块403记录该次写数据请求响应的开始时间。After the IO control module 402 sends a write data request to the memory 401, the IO statistics module 403 records the start time of the write data request response.
S590，内存401接收IO控制模块402发送的写数据请求，写入第一数据的日志记录，并向第一非易失性存储单元404发送上述写数据请求。S590, the memory 401 receives the write data request sent by the IO control module 402, writes a log record of the first data, and sends the write data request to the first non-volatile storage unit 404.
S5100，第一非易失性存储单元404接收内存401发送的写数据请求，写入第一数据的日志记录，向IO控制模块402返回成功写入的信息。S5100. The first non-volatile storage unit 404 receives the write data request sent by the memory 401, writes the log record of the first data, and returns the successfully written information to the IO control module 402.
具体地，第一非易失性存储单元404接收到内存401发送的写数据请求之后，判断第一非易失性存储单元404的写入速度是否不大于预设的阈值时，在不大于预设的阈值时，将第一数据的日志记录写入第一非易失性存储单元404。需要说明的是，当写入速度大于预设的阈值时，说明系统处于高负荷状态，如果再进行写请求处理，容易导致读写请求处理错误或者系统崩溃。此处，预设的阈值的大小根据设备当前进行读写请求处理的次数确定，在设备进行读写请求处理的次数比较大时，该预设的阈值可以设置长一些，但一般 均为毫秒级，相反，则该预设的阈值可以设置短一些。Specifically, after receiving the write data request sent by the memory 401, the first non-volatile storage unit 404 determines whether the write speed of the first non-volatile storage unit 404 is not greater than a preset threshold, and is not greater than When the threshold is set, the log record of the first data is written to the first nonvolatile storage unit 404. It should be noted that when the write speed is greater than the preset threshold, the system is in a high load state. If the write request processing is performed again, the read/write request processing error or the system crash may easily occur. Here, the preset threshold value is determined according to the number of times the device performs the read/write request processing. When the number of times the device performs the read/write request processing is relatively large, the preset threshold may be set longer, but generally All are in milliseconds. On the contrary, the preset threshold can be set shorter.
S5110，IO控制模块402将上述成功写入的信息返回给虚拟机，并向IO统计模块403发送所述第一数据的日志记录的大小。S5110, the IO control module 402 returns the successfully written information to the virtual machine, and sends the size of the log record of the first data to the IO statistics module 403.
S5120，IO统计模块403根据第一数据的日志记录的大小更新第一非易失性存储单元404的剩余空间。S5120, the IO statistics module 403 updates the remaining space of the first non-volatile storage unit 404 according to the size of the log record of the first data.
IO统计模块403在接收到第一数据的日志记录的大小之后，根据已记录的该次写数据请求响应的开始时间和接收到第一数据的日志记录的大小的时间，计算响应该次写数据请求的耗时，并根据第一数据的日志记录的大小更新内存401和第一非易失性存储单元404的剩余空间，举例为，分别减小记录的内存401和第一非易失性存储单元404的剩余空间，减小的值为第一数据的日志记录大小。After receiving the size of the log record of the first data, the IO statistics module 403 calculates the response to the write data according to the recorded start time of the write data request response and the time of receiving the log record of the first data. The time consuming of the request, and updating the remaining space of the memory 401 and the first non-volatile storage unit 404 according to the size of the log record of the first data, for example, respectively reducing the recorded memory 401 and the first non-volatile storage The remaining space of unit 404, the reduced value is the log record size of the first data.
上述IO控制模块402基于IO统计模块403对写数据请求的延时控制，可以达到在高速地响应写数据请求时，保证用户数据的不丢失。The above IO control module 402 can control the delay of the write data request based on the IO statistics module 403, so as to ensure that the user data is not lost when responding to the write data request at high speed.
本发明实施例提供的数据处理的方法，在判断第一数据的日志记录的大小大于第一非易失性存储单元404的剩余空间时，将第一非易失性存储单元404中的第二数据的日志记录转储于第二非易失性存储单元405中，在第一非易失性存储单元404的剩余空间足够大时，再向第一非易失性存储单元404写入数据，从而可以减小写入数据失败的概率。The data processing method provided by the embodiment of the present invention, when determining that the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 404, the second of the first non-volatile storage unit 404 The log record of the data is dumped in the second non-volatile storage unit 405, and when the remaining space of the first non-volatile storage unit 404 is sufficiently large, the data is written to the first non-volatile storage unit 404. Thereby the probability of failure to write data can be reduced.
图6为本发明实施例三提供的数据转储的方法流程图。如图6所示，所述方法具体包括：FIG. 6 is a flowchart of a method for data dumping according to Embodiment 3 of the present invention. As shown in FIG. 6, the method specifically includes:
S610，若第一数据的日志记录的大小大于第一非易失性存储单元404的剩余空间，IO统计模块403向所述IO控制模块402发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息。S610. If the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 404, the IO statistics module 403 sends data dump indication information to the IO control module 402, where the data dump indication information The address information of the second data is included.
IO统计模块403在向所述IO控制模块402发送数据转储指示信息之后，IO统计模块302记录该次数据转储请求响应的开始时间。After the IO statistics module 403 sends the data dump indication information to the IO control module 402, the IO statistics module 302 records the start time of the data dump request response.
S620，IO控制模块402根据所述第二数据的地址信息生成数据转储请求信 息，向所述第一非易失性存储单元404发送所述数据转储请求信息。S620. The IO control module 402 generates a data dump request signal according to the address information of the second data. And transmitting the data dump request information to the first non-volatile storage unit 404.
S630，第一非易失性存储单元404根据所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元405中。S630. The first non-volatile storage unit 404 dumps the log record of the second data in the second non-volatile storage unit 405 according to the data dump request information.
S640，第二非易失性存储单元405完成第二数据的日志记录写入之后，向第一非易失性存储单元404返回成功转储的信息。S640. After the second non-volatile storage unit 405 completes the log record writing of the second data, the information of the successful dump is returned to the first non-volatile storage unit 404.
S650，第一非易失性存储单元404接收到第二非易失性存储单元405返回的成功转储的信息之后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元404的剩余空间增加，并向IO控制模块402返回成功转储的信息。S650. After receiving the information of the successful dump returned by the second non-volatile storage unit 405, the first non-volatile storage unit 404 releases the storage space occupied by the log record of the second data, so that the first The remaining space of a non-volatile storage unit 404 is increased and the information of the successful dump is returned to the IO control module 402.
S660，IO控制模块402向IO统计模块403发送第二数据的日志记录的大小。S660, the IO control module 402 sends the size of the log record of the second data to the IO statistics module 403.
S670，IO统计模块403根据所述第二数据的日志记录的大小更新第一非易失性存储单元404和第二非易失性存储单元405的剩余空间。S670. The IO statistics module 403 updates the remaining space of the first nonvolatile storage unit 404 and the second nonvolatile storage unit 405 according to the size of the log record of the second data.
举例为，IO统计模块403根据已记录的该次数据转储请求响应的开始时间和接收到第二数据的日志记录的大小的时间，计算进行该次数据转储请求处理的耗时，增大记录的第一非易失性存储单元404的剩余空间，减小记录的第二非易失性存储单元405的剩余空间，增大和减小的值为第二数据的日志记录大小。For example, the IO statistic module 403 calculates the time taken to perform the data dump request processing according to the recorded start time of the data dump request response and the time of receiving the log record of the second data, and increases The remaining space of the recorded first nonvolatile storage unit 404 reduces the remaining space of the recorded second nonvolatile storage unit 405, and the increased and decreased values are the log record sizes of the second data.
上述在将第一非易失性存储单元404中的第二数据的日志记录转储于第二非易失性存储单元405的过程中，因为第一非易失性存储单元404中的数据同时在内存401也会存储，所以IO控制模块402可根据第一非易失性存储单元404的响应速度等指标，选择从第一非易失性存储单元404读取所需数据，或者直接从内存401中读取对应的数据。The above-described process of dumping the log record of the second data in the first non-volatile storage unit 404 in the second non-volatile storage unit 405 because the data in the first non-volatile storage unit 404 is simultaneously The memory 401 is also stored, so the IO control module 402 can select to read the required data from the first non-volatile storage unit 404 according to the response speed of the first non-volatile storage unit 404 or the like, or directly from the memory. The corresponding data is read in 401.
图7为本发明实施例四提供的数据处理设备的示意图，所述设备可以用于执行图2或图3所示的方法。图7中，所述数据处理设备包括：控制单元701，第一非易失性存储单元702和第二非易失性存储单元703，所述第一 非易失性存储单元702写数据的速度高于所述第二非易失性存储单元703写数据的速度。FIG. 7 is a schematic diagram of a data processing device according to Embodiment 4 of the present invention, where the device may be used to perform the method shown in FIG. 2 or FIG. 3. In FIG. 7, the data processing device includes: a control unit 701, a first non-volatile storage unit 702, and a second non-volatile storage unit 703, the first The non-volatile memory unit 702 writes data at a higher speed than the second non-volatile storage unit 703 writes data.
所述控制单元701，用于从虚拟机接收写数据请求，所述写数据请求包括第一数据。The control unit 701 is configured to receive a write data request from the virtual machine, where the write data request includes the first data.
确定所述第一数据的日志记录。Determining a log record of the first data.
比较所述第一数据的日志记录的大小与所述第一非易失性存储单元702的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元702的剩余空间，控制所述第一非易失性存储单元702将所述第一非易失性存储单元中702的第二数据的日志记录转储于所述第二非易失性存储单元703，以使所述第一非易失性存储单元702的剩余空间增加。Comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit 702, if the size of the log record of the first data is greater than the first non-volatile storage unit 702 Remaining space, controlling the first non-volatile storage unit 702 to dump a log record of the second data of the first non-volatile storage unit 702 to the second non-volatile storage unit 703 So that the remaining space of the first nonvolatile memory unit 702 is increased.
所述第一非易失性存储单元702，用于将所述第二数据的日志记录转储于所述第二非易失性存储单元703。The first non-volatile storage unit 702 is configured to dump the log record of the second data to the second non-volatile storage unit 703.
所述第二非易失性存储单元703，用于写入所述第二数据的日志记录。The second non-volatile storage unit 703 is configured to write a log record of the second data.
可选地，所述控制单元701包括IO控制模块704和IO统计模块705；Optionally, the control unit 701 includes an IO control module 704 and an IO statistics module 705;
所述IO控制模块704，用于将所述第一数据的日志记录大小发送给所述IO统计模块705；所述IO统计模块705保存所述第一非易失性存储单元702的剩余空间，用于比较所述IO控制模块704发送的第一数据的日志记录的大小与所述第一非易失性存储单元702的剩余空间，并向所述IO控制模块704返回比较结果信息。The IO control module 704 is configured to send a log record size of the first data to the IO statistic module 705; the IO statistic module 705 saves a remaining space of the first non-volatile storage unit 702. The size of the log record of the first data sent by the IO control module 704 is compared with the remaining space of the first non-volatile storage unit 702, and the comparison result information is returned to the IO control module 704.
可选地，所述IO统计模块705还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元702的剩余空间，则延迟指定阈值时间之后，再次比较所述第一数据的日志记录的大小与所述第一非易失性存储单元702的剩余空间。Optionally, the IO statistic module 705 is further configured to: after the specified threshold time is delayed, if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 702, compare the The size of the log record of the first data is the remaining space of the first non-volatile storage unit 702.
可选地，所述控制单元701包括IO控制模块704和IO统计模块705；Optionally, the control unit 701 includes an IO control module 704 and an IO statistics module 705;
所述IO控制模块704，用于向所述IO统计模块705发送查询请求信息，所述查询请求信息用于查询所述第一非易失性存储单元702的剩余空 间；从所述IO统计模块705获取所述第一非易失性存储单元702的剩余空间；比较所述第一数据的日志记录的大小与所述第一非易失性存储单元702的剩余空间。The IO control module 704 is configured to send query request information to the IO statistic module 705, where the query request information is used to query the remaining space of the first non-volatile storage unit 702. Obtaining a remaining space of the first non-volatile storage unit 702 from the IO statistic module 705; comparing a size of the log record of the first data with a remaining of the first non-volatile storage unit 702 space.
可选地，所述IO控制模块704还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元702的剩余空间，则延迟指定阈值时间之后，再次向所述IO统计模块705发送所述查询请求信息。Optionally, after the specified threshold time is delayed, The IO statistics module 705 sends the query request information.
可选地，所述IO统计模块705还用于向所述IO控制模块704发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息；所述IO控制模块704还用于根据所述IO统计模块705发送的第二数据的地址信息生成数据转储请求信息，向所述第一非易失性存储单元702发送所述数据转储请求信息；Optionally, the IO statistic module 705 is further configured to send data dump indication information to the IO control module 704, where the data dump indication information includes: address information of the second data; the IO control module The 704 is further configured to generate data dump request information according to the address information of the second data sent by the IO statistics module 705, and send the data dump request information to the first non-volatile storage unit 702.
所述第二非易失性存储单元703还用于向所述第一非易失存储单元702返回成功转储的信息；The second non-volatile storage unit 703 is further configured to return information that is successfully dumped to the first non-volatile storage unit 702;
所述第一非易失存储单元702还用于根据所述IO控制模块704发送的所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元703；在接收到所述第二非易失性存储单元703返回的所述成功转储的信息后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元702的剩余空间增加。The first nonvolatile storage unit 702 is further configured to dump the log record of the second data to the second nonvolatile storage according to the data dump request information sent by the IO control module 704. After the information about the successful dump returned by the second non-volatile storage unit 703 is received, the storage space occupied by the log record of the second data is released, so that the first non-easy The remaining space of the lossy memory unit 702 is increased.
可选地，所述IO控制模块704还用于根据所述成功转储的信息向所述IO统计模块705发送所述第二数据的日志记录的大小；所述IO统计模块705还用于根据所述IO控制模块704发送的所述第二数据的日志记录的大小更新所述第一非易失性存储单元702和所述第二非易失性存储单元703的剩余空间。Optionally, the IO control module 704 is further configured to send, according to the successfully dumped information, the size of the log record of the second data to the IO statistic module 705; the IO statistic module 705 is further configured to The size of the log record of the second data sent by the IO control module 704 updates the remaining space of the first non-volatile storage unit 702 and the second non-volatile storage unit 703.
可选地，所述设备还包括：写入单元706，用于若所述第一数据的日志记录的大小不大于所述第一非易失性存储单元702的剩余空间，并且所述第一非易失性存储单元702的写入速度不大于预设的阈值时，将所述第 一数据的日志记录写入所述第一非易失性存储单元702。Optionally, the device further includes: a writing unit 706, configured to: if a size of the log record of the first data is not greater than a remaining space of the first non-volatile storage unit 702, and the first When the writing speed of the nonvolatile memory unit 702 is not greater than a preset threshold, the first A log of data is written to the first non-volatile storage unit 702.
可选地，所述第一非易失性存储单元702还用于向所述IO控制模块704返回成功写入的信息；所述IO控制模块704还用于接收所述第一非易失性存储单元702返回的成功写入的信息，向所述IO统计模块705发送所述第一数据的日志记录的大小；Optionally, the first non-volatile storage unit 702 is further configured to return information that is successfully written to the IO control module 704; the IO control module 704 is further configured to receive the first non-volatile The successfully written information returned by the storage unit 702 is sent to the IO statistics module 705 for the size of the log record of the first data;
所述IO统计模块705还用于根据所述IO控制模块704发送的所述第一数据的日志记录的大小更新所述第一非易失性存储单元702的剩余空间。The IO statistic module 705 is further configured to update the remaining space of the first non-volatile storage unit 702 according to the size of the log record of the first data sent by the IO control module 704.
可选地，所述控制单元701，还用于从所述虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；根据所述待读出数据标识，从内存读取所述待读出数据；将所述待读出数据返回给所述虚拟机。本发明实施例四提供的设备植入了本发明实施例二提供的方法，因此，本发明提供的设备的具体工作过程，在此不复赘述。Optionally, the control unit 701 is further configured to receive a read data request from the virtual machine, where the read data request includes a data identifier to be read out, and read the memory from the memory according to the to-be-read data identifier. Data to be read; the data to be read is returned to the virtual machine. The device provided in the fourth embodiment of the present invention is embedded in the method provided in the second embodiment of the present invention. Therefore, the specific working process of the device provided by the present invention is not described herein.
本发明实施例提供的数据处理方法及设备，在判断第一数据的日志记录的大小大于第一非易失性存储单元702的剩余空间时，将第一非易失性存储单元702中的第二数据的日志记录转储于第二非易失性存储单元703中，在第一非易失性存储单元702的剩余空间足够大时，再向第一非易失性存储单元702写入数据，从而可以减小写入数据失败的概率。The data processing method and device provided by the embodiment of the present invention, when determining that the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit 702, the first non-volatile storage unit 702 The log record of the two data is dumped in the second non-volatile storage unit 703, and when the remaining space of the first non-volatile storage unit 702 is sufficiently large, the data is written to the first non-volatile storage unit 702. Thus, the probability of failure to write data can be reduced.
专业人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。A person skilled in the art should further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, in order to clearly illustrate hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
结合本文中所公开的实施例描述的方法或算法的步骤可以用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器 (RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage known in the art. In the medium.
以上所述的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。 The specific embodiments of the present invention have been described in detail with reference to the preferred embodiments of the present invention. All modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (20)

  1. 一种数据处理的方法，所述方法应用于数据处理设备，其特征在于，所述数据处理设备包括控制单元，第一非易失性存储单元和第二非易失性存储单元，所述第一非易失性存储单元写数据的速度高于所述第二非易失性存储单元写数据的速度，所述方法包括：A method of data processing, the method being applied to a data processing device, wherein the data processing device comprises a control unit, a first non-volatile storage unit and a second non-volatile storage unit, A non-volatile storage unit writes data at a higher speed than the second non-volatile storage unit writes data, the method comprising:
    所述控制单元从虚拟机接收写数据请求，所述写数据请求包括第一数据；The control unit receives a write data request from a virtual machine, the write data request including first data;
    确定所述第一数据的日志记录；Determining a log record of the first data;
    比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则所述控制单元控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据的日志记录转储于所述第二非易失性存储单元，以使所述第一非易失性存储单元的剩余空间增加。Comparing a size of the log record of the first data with a remaining space of the first non-volatile storage unit, if a size of the log record of the first data is greater than a remaining of the first non-volatile storage unit Space, the control unit controls the first non-volatile storage unit to dump a log record of the second data in the first non-volatile storage unit to the second non-volatile storage unit So that the remaining space of the first non-volatile memory unit is increased.
  2. 如权利要求1所述的方法，其特征在于，所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间之前，所述方法还包括：The method of claim 1, wherein the method further comprises: comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit, the method further comprising:
    所述控制单元的IO控制模块将所述第一数据的日志记录大小发送给所述控制单元的IO统计模块，所述IO统计模块保存所述第一非易失性存储单元的剩余空间；The IO control module of the control unit sends a log record size of the first data to an IO statistic module of the control unit, where the IO statistic module saves a remaining space of the first nonvolatile storage unit;
    所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间包括：The comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
    所述IO统计模块比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，并向所述IO控制模块返回比较结果信息。The IO statistic module compares the size of the log record of the first data with the remaining space of the first non-volatile storage unit, and returns comparison result information to the IO control module.
  3. 如权利要求2所述的方法，其特征在于，所述方法还包括： The method of claim 2, wherein the method further comprises:
    若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，所述IO统计模块再次比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。If the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, after the specified threshold time is delayed, the IO statistics module compares the size of the log record of the first data again. And a remaining space of the first nonvolatile storage unit.
  4. 如权利要求1所述的方法，其特征在于，所述比较所述第一数据的日志记录的大小与第一非易失性存储单元的剩余空间之前，所述方法还包括：The method of claim 1, wherein the method further comprises: comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit, the method further comprising:
    所述控制单元的IO控制模块向所述控制单元的IO统计模块发送查询请求信息，所述查询请求信息用于查询所述第一非易失性存储单元的剩余空间；The IO control module of the control unit sends query request information to the IO statistics module of the control unit, where the query request information is used to query the remaining space of the first non-volatile storage unit;
    所述IO控制模块从所述IO统计模块获取所述第一非易失性存储单元的剩余空间；The IO control module acquires a remaining space of the first non-volatile storage unit from the IO statistic module;
    所述比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间包括：The comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit includes:
    所述IO控制模块比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。The IO control module compares a size of the log record of the first data with a remaining space of the first non-volatile storage unit.
  5. 如权利要求4所述的方法，其特征在于，所述方法还包括：The method of claim 4, wherein the method further comprises:
    若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，所述IO控制模块再次向所述IO统计模块发送所述查询请求信息。If the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, after the specified threshold time is delayed, the IO control module sends the query request to the IO statistic module again. information.
  6. 如权利要求2或3所述的方法，其特征在于，所述若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，所述控制单元控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据转储于所述第二非易失性存储单元包括： The method according to claim 2 or 3, wherein said control unit controls said if said log data of said first data has a size larger than a remaining space of said first nonvolatile storage unit The dumping the second data in the first non-volatile storage unit to the second non-volatile storage unit by the first non-volatile storage unit includes:
    所述IO统计模块向所述IO控制模块发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息；The IO statistic module sends data dump indication information to the IO control module, where the data dump indication information includes: address information of the second data;
    所述IO控制模块根据所述第二数据的地址信息生成数据转储请求信息，向所述第一非易失性存储单元发送所述数据转储请求信息，以使所述第一非易失性存储单元根据所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元，并且在接收到所述第二非易失性存储单元返回的成功转储的信息后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元的剩余空间增加。The IO control module generates data dump request information according to the address information of the second data, and sends the data dump request information to the first non-volatile storage unit to enable the first non-volatile The storage unit dumps the log record of the second data to the second non-volatile storage unit according to the data dump request information, and receives the return of the second non-volatile storage unit After successfully dumping the information, the storage space occupied by the log record of the second data is released to increase the remaining space of the first non-volatile storage unit.
  7. 如权利要求6所述的方法，其特征在于，所述在接收到所述第二非易失性存储单元返回的成功转储的信息后，所述方法还包括：The method of claim 6, wherein after the receiving the information of the successful dump returned by the second non-volatile storage unit, the method further comprises:
    所述IO控制模块向所述IO统计模块发送所述第二数据的日志记录的大小，由所述IO统计模块根据所述第二数据的日志记录的大小更新所述第一非易失性存储单元和所述第二非易失性存储单元的剩余空间。The IO control module sends a size of the log record of the second data to the IO statistic module, and the IO statistic module updates the first non-volatile storage according to the size of the log record of the second data. a remaining space of the unit and the second non-volatile storage unit.
  8. 如权利要求1所述的方法，其特征在于，所述方法还包括：The method of claim 1 wherein the method further comprises:
    若所述第一数据的日志记录的大小不大于所述第一非易失性存储单元的剩余空间，并且所述第一非易失性存储单元的写入速度不大于预设的阈值时，将所述第一数据的日志记录写入所述第一非易失性存储单元。If the size of the log record of the first data is not greater than the remaining space of the first non-volatile storage unit, and the write speed of the first non-volatile storage unit is not greater than a preset threshold, Writing a log record of the first data to the first non-volatile storage unit.
  9. 如权利要求8所述的方法，其特征在于，所述在将所述第一数据的日志记录写入所述第一非易失性存储单元之后，所述方法还包括：The method of claim 8, wherein after the logging of the first data is written to the first non-volatile storage unit, the method further comprises:
    所述IO控制模块接收所述第一非易失性存储单元返回的成功写入的信息，向所述IO统计模块发送所述第一数据的日志记录的大小，由所述IO统计模块根据所述第一数据的日志记录的大小更新所述第一非易失性存储单元的剩余空间。 The IO control module receives the successfully written information returned by the first non-volatile storage unit, and sends the size of the log record of the first data to the IO statistics module, according to the IO statistics module. The size of the log record of the first data updates the remaining space of the first non-volatile storage unit.
  10. 如权利要求1-9任一所述的方法，其特征在于，所述方法还包括：The method of any of claims 1-9, wherein the method further comprises:
    所述控制单元从所述虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；The control unit receives a read data request from the virtual machine, and the read data request includes a data identifier to be read out;
    根据所述待读出数据标识，从所述内存读取所述待读出数据；Reading the data to be read from the memory according to the data identifier to be read;
    将所述待读出数据返回给所述虚拟机。Returning the data to be read to the virtual machine.
  11. 一种数据处理设备，其特征在于，所述数据处理设备包括：控制单元，第一非易失性存储单元和第二非易失性存储单元，所述第一非易失性存储单元写数据的速度高于所述第二非易失性存储单元写数据的速度；A data processing device, comprising: a control unit, a first non-volatile storage unit and a second non-volatile storage unit, wherein the first non-volatile storage unit writes data The speed is higher than the speed at which the second non-volatile storage unit writes data;
    所述控制单元，用于从虚拟机接收写数据请求，所述写数据请求包括第一数据；The control unit is configured to receive a write data request from a virtual machine, where the write data request includes first data;
    确定所述第一数据的日志记录；Determining a log record of the first data;
    比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，控制所述第一非易失性存储单元将所述第一非易失性存储单元中的第二数据的日志记录转储于所述第二非易失性存储单元，以使所述第一非易失性存储单元的剩余空间增加；Comparing a size of the log record of the first data with a remaining space of the first non-volatile storage unit, if a size of the log record of the first data is greater than a remaining of the first non-volatile storage unit Space, controlling the first non-volatile storage unit to dump a log record of the second data in the first non-volatile storage unit to the second non-volatile storage unit to cause the The remaining space of the first non-volatile memory unit is increased;
    所述第一非易失性存储单元，用于将所述第二数据的日志记录转储于所述第二非易失性存储单元；The first non-volatile storage unit is configured to dump the log record of the second data to the second non-volatile storage unit;
    所述第二非易失性存储单元，用于写入所述第二数据的日志记录。The second non-volatile storage unit is configured to write a log record of the second data.
  12. 如权利要求11所述的设备，其特征在于，所述控制单元包括IO控制模块和IO统计模块；The device according to claim 11, wherein said control unit comprises an IO control module and an IO statistic module;
    所述IO控制模块，用于将所述第一数据的日志记录大小发送给所述IO统计模块； The IO control module is configured to send a log record size of the first data to the IO statistic module;
    所述IO统计模块保存所述第一非易失性存储单元的剩余空间，用于比较所述IO控制模块发送的第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间，并向所述IO控制模块返回比较结果信息。The IO statistics module saves a remaining space of the first non-volatile storage unit, and compares a size of a log record of the first data sent by the IO control module with the first non-volatile storage unit. Remaining space and returning comparison result information to the IO control module.
  13. 如权利要求12所述的设备，其特征在于，所述IO统计模块还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，再次比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。The device according to claim 12, wherein the IO statistic module is further configured to: if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, delay designating After the threshold time, the size of the log record of the first data is compared with the remaining space of the first non-volatile storage unit.
  14. 如权利要求11所述的设备，其特征在于，所述控制单元包括IO控制模块和IO统计模块；The device according to claim 11, wherein said control unit comprises an IO control module and an IO statistic module;
    所述IO控制模块，用于向所述IO统计模块发送查询请求信息，所述查询请求信息用于查询所述第一非易失性存储单元的剩余空间；The IO control module is configured to send query request information to the IO statistic module, where the query request information is used to query a remaining space of the first non-volatile storage unit;
    从所述IO统计模块获取所述第一非易失性存储单元的剩余空间；Obtaining a remaining space of the first nonvolatile storage unit from the IO statistic module;
    比较所述第一数据的日志记录的大小与所述第一非易失性存储单元的剩余空间。Comparing the size of the log record of the first data with the remaining space of the first non-volatile storage unit.
  15. 如权利要求14所述的设备，其特征在于，所述IO控制模块还用于若所述第一数据的日志记录的大小大于所述第一非易失性存储单元的剩余空间，则延迟指定阈值时间之后，再次向所述IO统计模块发送所述查询请求信息。The device according to claim 14, wherein the IO control module is further configured to: if the size of the log record of the first data is greater than the remaining space of the first non-volatile storage unit, delay designating After the threshold time, the query request information is sent to the IO statistics module again.
  16. 如权利要求12或13所述的设备，其特征在于，The device according to claim 12 or 13, wherein
    所述IO统计模块，还用于向所述IO控制模块发送数据转储指示信息，所述数据转储指示信息包括：所述第二数据的地址信息；The IO statistic module is further configured to send data dump indication information to the IO control module, where the data dump indication information includes: address information of the second data;
    所述IO控制模块，还用于根据所述IO统计模块发送的第二数据的地址 信息生成数据转储请求信息，向所述第一非易失性存储单元发送所述数据转储请求信息；The IO control module is further configured to: according to the address of the second data sent by the IO statistics module Generating data dump request information, and transmitting the data dump request information to the first non-volatile storage unit;
    所述第二非易失性存储单元，还用于向所述第一非易失存储单元返回成功转储的信息；The second non-volatile storage unit is further configured to return information that is successfully dumped to the first non-volatile storage unit;
    所述第一非易失存储单元，还用于根据所述IO控制模块发送的所述数据转储请求信息将所述第二数据的日志记录转储于所述第二非易失性存储单元；在接收到所述第二非易失性存储单元返回的所述成功转储的信息后，释放所述第二数据的日志记录占用的存储空间，以使所述第一非易失性存储单元的剩余空间增加。The first nonvolatile storage unit is further configured to dump the log record of the second data to the second nonvolatile storage unit according to the data dump request information sent by the IO control module After receiving the successfully dumped information returned by the second non-volatile storage unit, releasing the storage space occupied by the log record of the second data, so that the first non-volatile storage The remaining space of the unit increases.
  17. 根据权利要求16所述的设备，其特征在于，The device of claim 16 wherein:
    所述IO控制模块，还用于根据所述成功转储的信息向所述IO统计模块发送所述第二数据的日志记录的大小；The IO control module is further configured to send, according to the successfully dumped information, a size of a log record of the second data to the IO statistic module;
    所述IO统计模块，还用于根据所述IO控制模块发送的所述第二数据的日志记录的大小更新所述第一非易失性存储单元和所述第二非易失性存储单元的剩余空间。The IO statistic module is further configured to update, according to the size of the log record of the second data sent by the IO control module, the first non-volatile storage unit and the second non-volatile storage unit remaining space.
  18. 根据权利要求11所述的设备，其特征在于，所述设备还包括：写入单元，用于若所述第一数据的日志记录的大小不大于所述第一非易失性存储单元的剩余空间，并且所述第一非易失性存储单元的写入速度不大于预设的阈值时，将所述第一数据的日志记录写入所述第一非易失性存储单元。The device according to claim 11, wherein the device further comprises: a writing unit, configured to: if a size of the log record of the first data is not greater than a remaining portion of the first non-volatile storage unit And storing a log record of the first data into the first non-volatile storage unit when a write speed of the first non-volatile storage unit is not greater than a preset threshold.
  19. 根据权利要求18所述的设备，其特征在于，The device of claim 18, wherein
    所述第一非易失性存储单元，还用于向所述IO控制模块返回成功写入的信息； The first non-volatile storage unit is further configured to return information that is successfully written to the IO control module;
    所述IO控制模块，还用于接收所述第一非易失性存储单元返回的成功写入的信息，向所述IO统计模块发送所述第一数据的日志记录的大小；The IO control module is further configured to receive information that is successfully written by the first non-volatile storage unit, and send a size of a log record of the first data to the IO statistics module;
    所述IO统计模块，还用于根据所述IO控制模块发送的所述第一数据的日志记录的大小更新所述第一非易失性存储单元的剩余空间。The IO statistic module is further configured to update a remaining space of the first non-volatile storage unit according to a size of a log record of the first data sent by the IO control module.
  20. 如权利要求11-19任一所述的设备，其特征在于，A device according to any of claims 11-19, wherein
    所述控制单元，还用于从所述虚拟机接收读数据请求，所述读数据请求包括待读出数据标识；The control unit is further configured to receive a read data request from the virtual machine, where the read data request includes a data identifier to be read out;
    根据所述待读出数据标识，从内存读取所述待读出数据；Reading the data to be read from the memory according to the data identifier to be read;
    将所述待读出数据返回给所述虚拟机。 Returning the data to be read to the virtual machine.
PCT/CN2014/089533 2013-12-19 2014-10-27 Data processing method and device WO2015090113A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310703014.5A CN103678149B (en) 2013-12-19 2013-12-19 Data processing method and device
CN201310703014.5 2013-12-19

Publications (1)

Publication Number Publication Date
WO2015090113A1 true WO2015090113A1 (en) 2015-06-25

Family

ID=50315780

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/089533 WO2015090113A1 (en) 2013-12-19 2014-10-27 Data processing method and device

Country Status (2)

Country Link
CN (1) CN103678149B (en)
WO (1) WO2015090113A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103678149B (en) * 2013-12-19 2017-01-18 华为技术有限公司 Data processing method and device
CN103838521B (en) * 2014-02-28 2017-02-08 华为技术有限公司 Data processing method and data processing device
CN104765576B (en) * 2015-04-28 2017-10-31 广东欧珀移动通信有限公司 A kind of date storage method and data storage device
CN105868093B (en) * 2016-04-25 2019-10-11 上海斐讯数据通信技术有限公司 A kind of log wiring method and server-side
CN106250322B (en) * 2016-08-12 2019-11-19 华为技术有限公司 A kind of method and apparatus for writing data
CN107766008A (en) * 2017-12-05 2018-03-06 珠海格力电器股份有限公司 Method and device for preventing abnormal writing of storage medium and electronic equipment
CN108710474A (en) * 2018-05-18 2018-10-26 郑州云海信息技术有限公司 A kind of date storage method, device and computer storage media
CN109918381B (en) * 2019-03-13 2021-06-11 北京百度网讯科技有限公司 Method and apparatus for storing data
CN110399101B (en) * 2019-07-19 2023-01-10 苏州浪潮智能科技有限公司 Write operation processing method, device and system of disk and storage medium
CN113885797B (en) * 2021-09-24 2023-12-22 济南浪潮数据技术有限公司 Data storage method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7373469B2 (en) * 2005-08-26 2008-05-13 Hitachi, Ltd. Data migration method
US7685186B2 (en) * 2007-06-25 2010-03-23 Microsoft Corporation Optimized and robust in-place data transformation
CN102075670A (en) * 2009-11-24 2011-05-25 新奥特（北京）视频技术有限公司 Log recording method and device for broadcast machine
CN103345368A (en) * 2013-07-18 2013-10-09 四川九成信息技术有限公司 Data caching method in buffer storage
CN103678149A (en) * 2013-12-19 2014-03-26 华为技术有限公司 Data processing method and device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102184141A (en) * 2011-05-05 2011-09-14 曙光信息产业(北京)有限公司 Method and device for storing check point data
CN103412825B (en) * 2013-05-21 2016-12-28 青岛海信移动通信技术股份有限公司 A kind of memory release method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7373469B2 (en) * 2005-08-26 2008-05-13 Hitachi, Ltd. Data migration method
US7685186B2 (en) * 2007-06-25 2010-03-23 Microsoft Corporation Optimized and robust in-place data transformation
CN102075670A (en) * 2009-11-24 2011-05-25 新奥特（北京）视频技术有限公司 Log recording method and device for broadcast machine
CN103345368A (en) * 2013-07-18 2013-10-09 四川九成信息技术有限公司 Data caching method in buffer storage
CN103678149A (en) * 2013-12-19 2014-03-26 华为技术有限公司 Data processing method and device

Also Published As

Publication number Publication date
CN103678149A (en) 2014-03-26
CN103678149B (en) 2017-01-18

Similar Documents

Publication Publication Date Title
WO2015090113A1 (en) Data processing method and device
US10318324B2 (en) Virtualization support for storage devices
US9684460B1 (en) Proactively correcting behavior that may affect I/O performance in a non-volatile semiconductor storage device
US11243706B2 (en) Fragment management method and fragment management apparatus
US9588699B1 (en) Scheduling of reactive I/O operations in a storage environment
US9317320B2 (en) Hypervisor-based server duplication system and method and storage medium storing server duplication computer program
US8862820B2 (en) Scheduling of reconstructive I/O read operations in a storage environment
KR20200113047A (en) Operation method of open-channel storage device
US8849966B2 (en) Server image capacity optimization
JP6652647B2 (en) Storage system
US10031689B2 (en) Stream management for storage devices
US11416403B2 (en) Method and apparatus for performing pipeline-based accessing management in storage server with aid of caching metadata with hardware pipeline module during processing object write command
JP4865075B1 (en) Computer and computer system
US10956245B1 (en) Storage system with host-directed error scanning of solid-state storage devices
US10671525B2 (en) Space reclamation in data deduplication storage systems
US11074186B1 (en) Logical management of a destage process and dynamic cache size of a tiered data storage system cache that is configured to be powered by a temporary power source during a power loss event
US10394673B2 (en) Method and system for hardware accelerated copyback
JP5012599B2 (en) Memory content restoration device, memory content restoration method, and memory content restoration program
CN112860599A (en) Data caching processing method and device and storage medium
US20230315316A1 (en) Reading a master boot record for a namespace after reformatting the namespace
US8909827B2 (en) Method to allow a host to replace logical path resources it owns
US20140195732A1 (en) METHOD AND SYSTEM TO MAINTAIN MAXIMUM PERFORMANCE LEVELS IN ALL DISK GROUPS BY USING CONTROLLER VDs FOR BACKGROUND TASKS

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14871856

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14871856

Country of ref document: EP

Kind code of ref document: A1