CN113031869B - Data processing method and device and computer readable storage medium - Google Patents
Data processing method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN113031869B CN113031869B CN202110322457.4A CN202110322457A CN113031869B CN 113031869 B CN113031869 B CN 113031869B CN 202110322457 A CN202110322457 A CN 202110322457A CN 113031869 B CN113031869 B CN 113031869B
- Authority
- CN
- China
- Prior art keywords
- data
- disk
- target
- written
- check
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1048—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using arrangements adapted for a specific error detection or correction feature
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a data processing method, which comprises the following steps: acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written; determining a target data disk from the plurality of data disks based on the position information and the size of the data to be written; and writing the data to be written into the target data disk through the target data disk. The embodiment of the application also discloses a second electronic device and a computer readable storage medium.
Description
Technical Field
The present invention relates to data processing technologies in the field of computers, and in particular, to a data processing method, a data processing device, and a computer-readable storage medium.
Background
At present, erasure code technology is used in distributed storage systems to save storage space and greatly reduce cost under the condition of realizing availability of the systems. However, when data is written in a distributed storage system by using an erasure code technology, data is processed by a main data disk, and the processed data can be written in by a corresponding data disk after the main data disk processes the data; this process in the related art greatly increases the load on the main data disk, which in turn results in lower data writing efficiency.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application are expected to provide a data processing method, a device, and a computer-readable storage medium, which solve the problems of large pressure of a main data disk and low data writing efficiency in the related art.
In order to achieve the purpose, the technical scheme of the application is realized as follows:
a method of data processing, the method comprising:
acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written;
determining a target data disk from a plurality of data disks based on the position information and the size of the data to be written;
and writing the data to be written into the target data disk through the target data disk.
In the foregoing solution, the writing the data to be written into the target data disk by using the target data disk includes:
under the condition that the plurality of data disks do not have first data, writing the data to be written into the target data disk through the target data disk;
processing the data to be written through the target check disk, and determining a first check value corresponding to the data to be written;
and writing the first check value into the target check disk through the target check disk.
In the foregoing solution, the writing the data to be written into the target data disk by the target data disk includes:
determining, by the target data disk, whether there is second data in the target data disk if there is first data in the plurality of data disks;
under the condition that the target data disk does not have the second data, writing the data to be written into the target data disk through the target data disk;
updating a check value in the target check disk based on the data to be written through the target check disk; and the check value in the target check disk is the check value corresponding to the first data.
In the foregoing solution, the method further includes:
under the condition that the target data disk has the second data, writing the data to be written into the target data disk through the target data disk, and determining incremental data based on the data to be written and the second data;
and updating the check value in the target check disk based on the incremental data through the target check disk.
In the foregoing solution, the updating, by the target verification disk, the verification value in the target verification disk based on the data to be written includes:
and updating the second check value based on the data to be written and the second check value in the target check disk through the target check disk.
In the foregoing solution, the updating, by the target check disk, the second check value based on the data to be written and the second check value in the target check disk includes:
determining a third check value based on the data to be written and a second check value in the target check disk through the target check disk;
and replacing the second check value in the target check disk based on the third check value through the target check disk.
In the foregoing solution, the updating, by the target verification disk, the verification value in the target verification disk based on the incremental data includes:
and updating a fourth check value based on the incremental data and the fourth check value in the target check disk through the target check disk.
In the foregoing solution, the updating, by the target verification disk, the fourth verification value based on the incremental data and the fourth verification value in the target verification disk includes:
determining a fifth check value based on the incremental data and a fourth check value in the target check disk through the target check disk;
and replacing, by the target check disk, a fourth check value in the target check disk based on the fifth check value.
In the foregoing solution, the acquiring data writing information includes:
receiving a data writing request sent by first electronic equipment, and analyzing the data writing request to obtain data writing information;
correspondingly, the method further comprises the following steps:
monitoring whether the data to be written is successfully written into the target data disk or not through the target data disk;
generating feedback information based on a monitoring result and sending the feedback information to the first electronic equipment through the target data disk; the feedback information is used for indicating whether the first electronic device reinitiates the data writing request.
A second electronic device, the second electronic device comprising: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is configured to execute the data processing program stored in the memory to implement the steps of:
acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written;
determining a target data disk from a plurality of data disks based on the position information and the size of the data to be written;
and writing the data to be written into the target data disk through the target data disk.
A computer readable storage medium stores one or more programs, which are executable by one or more processors, to implement the steps of the above-described data processing method.
The data processing method, the data processing equipment and the computer readable storage medium provided by the embodiment of the application acquire data write-in information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written; determining a target data disk from the plurality of data disks based on the position information and the size of the data to be written; writing the data to be written into the target data disk through the target data disk; therefore, the data to be written can be directly written into the corresponding target data disk according to the writing position in the data writing information without selecting the main data disk, and the main data disk determines the data disk into which the data to be written is written, so that the pressure of the main data disk is released, the data writing process is optimized, and the data writing efficiency is improved.
Drawings
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a schematic diagram illustrating an application of erasure codes in a data processing method according to an embodiment of the present application;
fig. 3 is a schematic flow chart of another data processing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart of another data processing method provided in the embodiment of the present application;
FIG. 5 is a schematic flowchart illustrating a new write operation in a data processing method according to an embodiment of the present application;
FIG. 6 is a schematic flowchart illustrating an overwrite operation in a data processing method according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a second electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
An embodiment of the present application provides a data processing method, which is applied to a second electronic device, and as shown in fig. 1, the method includes the following steps:
s101, acquiring data writing information.
The data writing information comprises data to be written and position information corresponding to the data to be written; the location information characterizes a writing location of the data to be written.
In this embodiment of the application, the second electronic device may be a device having data processing and storing functions, and the data writing information may be information received by the second electronic device and sent by the first electronic device; the second electronic device can detect data transmitted on the data bus in real time, and when the second electronic device detects data write-in information, the data write-in information is acquired.
In a possible implementation manner, after the second electronic device obtains the data writing information, the information type of the data writing information may be determined based on the identification information of the data writing information; if the information type of the data writing information is consistent with the information type stored in the second electronic equipment in advance, the second electronic equipment performs subsequent processing on the data to be written in the acquired data writing information; and if the information type of the data writing information is inconsistent with the preset stored information type, the second electronic equipment does not perform any processing on the data to be written.
In the embodiment of the application, the second electronic device can analyze the data writing information and acquire the data to be written and the writing position of the data to be written from the data writing information; in a possible implementation manner, the size of the data to be written may be 1k of data, where the length of the data to be written may be represented by 0 to 1k, and the writing position of the data to be written may be a certain data disk of the multiple data disks. S102, determining a target data disk from the plurality of data disks based on the position information and the size of the data to be written.
In this embodiment of the present application, the second electronic device may include a plurality of data disks, where each data disk is a disk having data processing and computing functions; wherein the location information may include location information of at least one data disk; the location information may include, but is not limited to, identification information of the data disk; the number of target data disks may be one or more.
Specifically, the second electronic device may divide the data to be written into at least one data block based on the size of the data to be written, and determine a target data disk corresponding to each data block based on the at least one data block and the location information. If the number of the data blocks is multiple, a certain arrangement sequence is formed among the multiple data blocks; wherein the number of data blocks corresponds to the number of target data disks.
It should be noted that the second electronic device may process the data to be written by using an erasure correction coding technique, where the erasure correction coding technique mainly divides the data to be written into n block data blocks by using an erasure correction coding algorithm, and calculates the n block data blocks by using an erasure correction coding matrix to obtain m check blocks (check data); wherein, the set composed of n data blocks and m check blocks is called a stripe. Wherein, the erasure code has a plurality of modes, and the erasure code mode at least includes: "2+1 mode" and "4+2 mode"; in the 2+1 mode, "2" indicates that the number of data blocks in each stripe is "2", and "1" indicates that the number of parity blocks in each stripe is "1";4+2 mode, "4" means the number of data blocks in each stripe is "4", and "2" means the number of parity blocks in each stripe is "2"; the sizes of the erasure code data block and the erasure code check block in the distributed storage system can be set to be 4K.
As shown in fig. 2, the erasure code mode is "2+1 mode", and when the data to be written is 16K data, the second electronic processing device divides the data to be written into 4 data blocks by using "2+1 mode" of the erasure code, and each stripe has two data blocks and one check block in total; the check block (check data) in each stripe is obtained by checking two data blocks in the stripe; the size of each stripe is the number of data blocks in each stripe, and the width of each stripe is the size of the data blocks in the stripe; the size of the stripe in the "2+1 pattern" for erasure codes is "2". In a distributed storage system, a data block may be written to a data disk. That is, as shown in fig. 2, the second electronic device needs to use 4 data disks to write 16k of data to be written, and also needs 2 check disks to write 2 check blocks.
In this embodiment of the application, after the second electronic device obtains the data to be written from the data writing information, the second electronic device may divide the data to be written into at least one data block based on the size of the data to be written by using an erasure code technique, and determine a target data disk for the data to be written based on a division result and the location information.
In a feasible implementation manner, if the size of the data to be written is 1K, and the data length is 0-1K, the location information may be identification information of a disk a; the second electronic device divides data to be written with the data length of 0-1k into 1 data block by adopting an erasure code technology, determines a disk A in the multiple data disks based on identification information of the disk A, and takes the disk A as a target data disk corresponding to the data block.
In another feasible implementation manner, if the size of the data to be written is 8K, the data length is 0-8K, the writing position of the data to be written may be identification information of an a disk and a B disk, the second electronic device divides the data to be written of 8K into 2 data blocks by using an erasure code technology, where the data corresponding to the 1 st data block is 0-4K of the data to be written, the data corresponding to the 2 nd data block is 5K-8K of the data to be written, the second electronic device may determine the a disk and the B disk from among the multiple data disks based on the identification information of the a disk and the B disk, and then the second electronic device may determine a target data disk corresponding to the 1 st data block and a target data disk corresponding to the 2 nd data block from the a disk and the B disk. Wherein the number of identification information corresponds to the number of target data disks.
And S103, writing the data to be written into the target data disk through the target data disk.
In the embodiment of the application, the modes for writing the data to be written into the target data disk comprise initial writing and updating writing; the initial writing refers to a mode of writing data to be written into a target data disk when all data disks of the second electronic equipment have no data; update writing refers to the manner of writing when data is present in any data disk. Writing the data to be written into the target data disk comprises a writing process and a verification process. The writing process refers to directly writing the data to be written into a target data disk; the verification process refers to a process of verifying data to be written.
The data processing method provided by the embodiment of the application acquires data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written; determining a target data disk from the plurality of data disks based on the position information and the size of the data to be written; writing the data to be written into the target data disk through the target data disk; therefore, the data to be written can be directly written into the corresponding target data disk according to the writing position in the data writing information without selecting the main data disk, and the main data disk determines the data disk into which the data to be written is written, so that the pressure of the main data disk is released, the flow of writing the data to be written is optimized, and the data writing efficiency is improved.
Based on the foregoing embodiments, an embodiment of the present application provides a data processing method, which is shown in fig. 3 and includes the following steps:
s201, the second electronic equipment acquires data writing information.
The data writing information comprises data to be written and position information corresponding to the data to be written; the location information characterizes a writing location of the data to be written.
S202, the second electronic equipment determines a target data disk from the plurality of data disks based on the position information and the size of the data to be written.
S203, the second electronic device writes the data to be written into the target data disk through the target data disk under the condition that the plurality of data disks do not have the first data.
In this embodiment of the application, when the plurality of data disks do not have the first data, it means that all the data disks do not have data, and at this time, the target data disk may write the data to be written into the target data disk.
And S204, the second electronic device processes the data to be written through the target check disk, and determines a first check value corresponding to the data to be written.
And the check value in the target check disk is the check value corresponding to the first data.
In this embodiment of the application, when the plurality of data disks do not have the first data, the target verification disk does not have the data, and at this time, the target verification disk may verify the data to be written to generate the first verification value. The target check disk may be a check disk corresponding to the target data disk.
It should be noted that, in the related art, the zero padding operation and the calculation of the first check value for the data to be written are both performed by one data disk; in the embodiment of the application, the first check value is directly calculated by the target check disk, that is, the data disk for writing the data to be written and the check disk for calculating the first check value are not the same data disk, so that the load when one data disk processes the data to be written is reduced.
S205, the second electronic device writes the first check value into the target check disk through the target check disk.
In the embodiment of the application, the target check disk writes (stores) the first check value obtained by self calculation into the target check disk, and the target check disk does not need to acquire the first check value from the main data disk any more, so that the problem that the target check disk cannot acquire the first check value from the main data disk when the main data disk fails or the network fails is avoided, the processing of data to be written is influenced, and the load capacity and the network overhead of the main data disk are reduced.
According to the data processing method provided by the embodiment of the application, the data to be written can be directly written into the corresponding target data disk according to the writing position in the data writing information, the main data disk does not need to be selected, the main data disk determines the data disk into which the data to be written is written, the pressure of the main data disk is released, the flow of writing the data to be written is optimized, and the data writing efficiency is improved.
Based on the foregoing embodiments, an embodiment of the present application provides a data processing method, which is shown in fig. 4 and includes the following steps:
s301, the second electronic equipment acquires data writing information.
S302, the second electronic device determines a target data disk from the plurality of data disks based on the position information and the size of the data to be written.
And S303, under the condition that the second electronic equipment has the first data in the plurality of data disks, determining whether the target data disk has the second data or not through the target data disk.
In the embodiment of the application, under the condition that a plurality of data disks have first data, the target data disk can read data stored by the target data disk to determine whether the target data disk has second data, and if the target data disk does not read the data stored by the target data disk, the target data disk does not have the second data; and if the target data disk reads the data stored in the target data disk, indicating that the target data disk has the second data.
It should be noted that, whether to perform a new write operation or an overwrite write operation on the data to be written is determined by determining whether the target data disk has the second data. Wherein the new write operation indicates an operation performed when there is no data in the target data disk; the overwrite write operation indicates an operation to be performed when data is present in the target data disk.
In this embodiment of the present application, when the target data disk executes a new write operation, the following steps S304 and S305 may be executed; when the target data disk performs the overwrite operation, the following steps of S306, S307, and S308 may be performed.
S304, under the condition that the second data does not exist in the target data disk, the second electronic equipment writes the data to be written into the target data disk through the target data disk.
In this embodiment of the application, when the target data disk does not have the second data, a new write operation may be performed on the data to be written, and the target data disk may directly write the data to be written into the target data disk. The new write operation and the overwrite write operation both comprise a write-in process of data to be written and a verification process of the data to be written. The write flow of the data to be written is the same whether a new write operation or an overwrite write operation is performed.
S305, the second electronic device updates the check value in the target check disk based on the data to be written through the target check disk.
And the check value in the target check disk is the check value corresponding to the first data.
It should be noted that, in S305, a verification process of data to be written of the new write operation is performed. Because the target check disk has no data, the data to be written can be used as incremental data, and the target data disk can update the check value in the target check disk based on the data to be written. The incremental data refers to difference data between the data to be written and second data of the target check disk.
S305 may be executed by S305A in the embodiment of the present application.
S305A, the second electronic device updates a second check value based on the data to be written and the second check value in the target check disk through the target check disk.
It should be noted that, in the embodiment of the present application, a plurality of data disks correspond to one check disk, and therefore, the check value in the check disk is not only the check value in one data disk, and for example, when the mode of the erasure code is "2+1 mode", it indicates that 2 data disks correspond to one check disk. That is, although there is no data in the target data disk, there may still be check values of data in other data disks in the target check disk.
In this embodiment of the present application, no matter whether the target check disk has the check value, the target check disk may obtain the second check value in the target check disk according to the size of the data to be written, and update the second check value.
In a feasible implementation manner, when the size of the data to be written is 1k, the length of the data to be written may be represented as 0-1k, at this time, if the target check disk already stores a check value (check data) of 4k, and the data length of the check value of 4k is 0-4k, the target check disk may read the check value corresponding to the data length of 0-1k in the target check disk, and use the check value as a second check value, and then only update the check value corresponding to the data length of 0-1k in the target check disk based on the data to be written and the second check value.
In another possible implementation manner, if the target check disk has no data, the target check disk cannot read the data (check value) in the target check disk, which indicates that the second check value is null, and the target check disk may update the check value in the target check disk based on the data to be written.
In this embodiment, S305A may be implemented by S305A and S305 b;
s305a, the second electronic device determines a third check value based on the data to be written and the second check value in the target check disk through the target check disk.
In this embodiment of the present application, when the target check disk updates the second check value, a third check value may be first calculated according to the data to be written and the second check value, and then the second check value is updated according to the third check value.
In a feasible implementation manner, if the data to be written is data with a data length of 0 to 1k and is recorded as data a, the second check value is data with a data length of 0 to 1k in the target check disk and is recorded as data b, the target check disk may recalculate the check data with a data length of 0 to 1k based on the data a and the data b and is recorded as data c, and the data c is used as a third check value.
And S305b, replacing the second check value in the target check disk by the second electronic device through the target check disk based on the third check value.
In this embodiment of the present application, the second check value in the replacement target check disk may refer to a part of check values in the replacement target check disk, and may also refer to all check values in the replacement target check disk.
In a feasible implementation manner, if the size of the data block is a and the data to be written is data with a data length of 0-a, the replaced second parity value is all data (all parity values) in the target parity disk; if the data to be written is data with a data length of 0-B and B is smaller than a, the replaced second check value is a partial check value (partial data) in the target check disk.
It should be noted that, whether a part of check values in the replacement target check disk or all check values in the replacement target check disk are determined according to the data length of the data to be written; compared with the prior art that all the check values in the target check disk are directly updated, the complexity of updating is reduced.
The following describes in detail a write flow and a verification flow for writing data to be written into the target data disk by the target data disk in the case where the target data disk does not have the second data, with reference to fig. 5.
As shown in fig. 5, the first electronic device sends data to be written with a data size of 1k to a target data disk in the second electronic device, at this time, the target data disk may send the data to be written to a target verification disk, the target verification disk calculates a new verification value based on the data to be written and verification data corresponding to the data length of 0-1k in the target verification disk, and updates data corresponding to the data length of 0-1k in the target verification disk (an old verification value corresponding to 0-1 k) based on the new verification value.
S306, under the condition that the second data exist in the target data disk, the second electronic device writes the data to be written into the target data disk through the target data disk, and determines incremental data based on the data to be written and the second data.
In the embodiment of the application, under the condition that the target data disk has the second data, the overwriting operation can be executed on the data to be written; the target data disk may perform a write procedure that overwrites data to be written in the write operation and determines incremental data. The incremental data refers to difference data between the data to be written and second data of the target check disk.
In a feasible implementation manner, if the size of the second data in the target data disk is 4k, the data length of the second data may be 0-4k, the size of the data to be written is 1k, and the length of the data to be written may be 0-1k, the target data disk may read the second data with the data length of 0-1k in the target data disk based on the length of the data to be written, and determine incremental data in the data to be written with the data length of 0-1k based on the second data with the data length of 0-1k and the data to be written with the data length of 0-1 k; if the data to be written with the data length of 0-1k is completely different from the second data with the data length of 0-1k, the incremental data is the data to be written with the data length of 0-1 k; if the data to be written with the data length of 0-1k is the same as part of the data in the second data with the data length of 0-1k, determining data different from the second data with the data length of 0-1k from the data to be written with the data length of 0-1k, and taking the different data as incremental data.
And S307, the second electronic device updates the check value in the target check disk based on the incremental data through the target check disk.
It should be noted that, in S307, a verification process is executed to overwrite data to be written in the write operation, and the target verification disk may update the verification value in the target verification disk based on the length of the data to be written and the incremental data.
S307 may also be implemented by S307A in this embodiment,
and S307A, the second electronic device updates the fourth check value based on the incremental data and the fourth check value in the target check disk through the target check disk.
And the fourth check value refers to a check value in a target check disk corresponding to the data to be written.
In a feasible implementation manner, if the data length of the data to be written is 0 to 1k, and the data increment is 0.5 to 0.8k of data in the data length of the data to be written, the target check disk may read a check value with a data length of 0 to 1k from the target check disk based on the length of the data to be written, calculate a new check value with a data length of 0 to 1k based on the check value with the data length of 0 to 1k and the data with the data length of 0.5 to 0.8k of the data length of the data to be written, and then update the read check value with the data length of 0 to 1k by using the new check value with the data length of 0 to 1 k.
In the embodiment of the present application, S307A may be specifically implemented by S307A and S307 b;
and S307a, the second electronic device determines a fifth check value based on the incremental data and the fourth check value in the target check disk through the target check disk.
In a feasible implementation manner, if the data to be written is data with a data length of 0 to 2k, denoted as data m, the incremental data is data with a data length of 0.5 to 0.9k in the data to be written, denoted as data n, the fourth check value is data with a data length of 0 to 2k in the target check disk, and the target check disk may recalculate data with a data length of 0 to 2k based on the data m and the data n, and use the data with a data length of 0 to 2k as the fifth check value.
And S307b, the second electronic device replaces the fourth verification value in the target verification disk based on the fifth verification value through the target verification disk.
It should be noted that the implementation processes of S307b and S305b are the same, and are not described herein again.
A detailed explanation will be given below with reference to fig. 6 on a write flow and a verification flow for writing data to be written into the target data disk by the target data disk in the case where the target data disk has the second data.
As shown in fig. 6, the first electronic device sends data to be written with a data size of 1k to a target data disk in the second electronic device, where the data length of the data to be written is 0-1k, at this time, the target data disk may send incremental data determined based on data with a data length of 0-1k (old 1k data) and 1k data to be written in the target data disk, write the data to be written of 1k into the target data disk, then the target data disk sends the incremental data to a target verification disk, the target verification disk calculates a new verification value based on the incremental data and a verification value with a data length of 0-1k (old verification value) in the target verification disk, and replaces the verification value with the new verification value, where the data length of 0-1k in the target verification disk.
Based on the foregoing embodiments, in other embodiments of the present application, S201 and S301 may be implemented by S308:
s308, receiving a data writing request sent by the first electronic device, and analyzing the data writing request to obtain the data writing information.
In this embodiment of the application, after the first electronic device may send a data write request to the second electronic device, the second electronic device may detect the data write request, and after detecting the data write request, analyze the data write request to obtain data write information.
Based on the foregoing embodiments, in other embodiments of the present application, the method may further include the following steps:
s309, the second electronic device monitors whether the data to be written is successfully written into the target data disk through the target data disk.
In the embodiment of the present application, determining whether the data to be written is written successfully may be implemented by determining whether the data to be written is written in the target data disk and whether the check value is updated in the target check disk. And determining that the data to be written is successfully written into the target data disk only when the data to be written is successfully written into the target data disk and the check value in the target check disk is updated. And if the data to be written fails to be written and/or the check value in the target check disk is not updated, determining that the data to be written fails to be written into the target data disk.
Specifically, the target data disk compares data in the target data disk with data to be written in by reading the data in the target data disk, and if the target data disk has the data to be written in, it is determined that the data to be written in is written in successfully; if the target data disk does not have the data to be written, determining that the data to be written fails to be written; and the target check disk reads the check value in the target check disk based on the data to be written, determines whether part of the check value in the target check disk corresponding to the data to be written changes, and determines that the check value in the target check disk is updated if the part of the check value changes.
And S310, the second electronic equipment generates feedback information based on the monitoring result through the target data disk and sends the feedback information to the electronic equipment.
The feedback information is used for indicating the processing result of the data writing request queried by the electronic equipment.
In the embodiment of the application, the feedback information is used to indicate whether the data write request is successfully processed, and if the feedback information indicates that the data write request is unsuccessfully processed, the second electronic device may initiatively reinitiate the data write request; if the feedback information indicates that the data write request processing is successful, the second electronic device may generate prompt information based on the feedback information and display the prompt information.
According to the data processing method provided by the embodiment of the application, the data to be written can be directly written into the corresponding target data disk according to the writing position in the data writing information, the main data disk does not need to be selected, the main data disk determines the data disk into which the data to be written is written, the pressure of the main data disk is released, the flow of writing the data to be written is optimized, and the data writing efficiency is improved.
Based on the foregoing embodiments, an embodiment of the present application provides a second electronic device, where the second electronic device may be applied to the data processing method provided in the embodiments corresponding to fig. 1, 3, and 4, and as shown in fig. 7, the second electronic device 4 includes: a memory 41, a processor 42, and a communication bus 43;
the communication bus 43 is used for realizing communication connection between the processor 42 and the memory 41;
the processor 42 is configured to execute the data processing program stored in the memory 41 to implement the following steps:
acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written;
determining a target data disk from the plurality of data disks based on the position information and the size of the data to be written;
and writing the data to be written into the target data disk through the target data disk.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
under the condition that the plurality of data disks do not have the first data, writing the data to be written into the target data disk through the target data disk;
processing data to be written through a target check disk, and determining a first check value corresponding to the data to be written; the check value in the target check disk is a check value corresponding to the first data;
and writing the first check value into the target check disk through the target check disk.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
determining whether the target data disk has second data or not through the target data disk under the condition that the plurality of data disks have the first data;
under the condition that the target data disk does not have the second data, writing the data to be written into the target data disk through the target data disk;
updating a check value in the target check disk based on the data to be written through the target check disk; and the check value in the target check disk is the check value corresponding to the first data.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
under the condition that the target data disk has second data, writing data to be written into the target data disk through the target data disk, and determining incremental data based on the data to be written and the second data;
and updating the check value in the target check disk based on the incremental data through the target check disk.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
and updating the second check value based on the data to be written and the second check value in the target check disk through the target check disk.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
determining a third check value based on the data to be written and the second check value in the target check disk through the target check disk;
and replacing the second check value in the target check disk through the target check disk based on the third check value.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
and updating the fourth check value based on the incremental data and the fourth check value in the target check disk through the target check disk.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
determining a fifth check value based on the incremental data and the fourth check value in the target check disk through the target check disk;
and replacing, by the target check disk, the fourth check value in the target check disk based on the fifth check value.
In other embodiments of the present application, the processor 42 is configured to execute the executable instructions stored in the memory 41 to implement the following steps:
receiving a data writing request sent by the electronic equipment, and analyzing the data writing request to obtain data writing information;
monitoring whether the data to be written is successfully written into the target data disk or not through the target data disk;
generating feedback information based on the monitoring result through the target data disk and sending the feedback information to the electronic equipment; the feedback information is used for indicating the processing result of the data writing request inquired by the electronic equipment.
It should be noted that, in the embodiment of the present application, a specific implementation process of the step executed by the processor 42 may refer to an implementation process in the data processing method provided in the embodiments corresponding to fig. 1, 3, and 4, and is not described herein again.
The second electronic device provided by the embodiment of the application can directly write the data to be written into the corresponding target data disk according to the writing position in the data writing information without selecting the main data disk, and the main data disk determines the data disk into which the data to be written is written, so that the pressure of the main data disk is released, the flow of writing the data to be written is optimized, and the data writing efficiency is improved.
Based on the foregoing embodiments, the present application provides a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement the steps in the data processing method provided by the embodiments corresponding to fig. 1, 3 and 4.
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The computer-readable storage medium may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM), and the like; and may be various electronic devices such as mobile phones, computers, tablet devices, personal digital assistants, etc., including one or any combination of the above-mentioned memories.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method described in the embodiments of the present invention.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method of data processing, the method comprising:
acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written;
determining a target data disk from a plurality of data disks based on the position information and the size of the data to be written;
writing the data to be written into the target data disk through the target data disk;
wherein writing the data to be written to the target data disk by the target data disk includes:
under the condition that the plurality of data disks have first data and the target data disk has second data, writing the data to be written into the target data disk through the target data disk, and determining incremental data based on the data to be written and the second data;
and updating the check value in the target check disk based on the incremental data through the target check disk.
2. The method of claim 1, wherein the writing the data to be written to the target data disk by the target data disk comprises:
under the condition that the plurality of data disks do not have first data, writing the data to be written into the target data disk through the target data disk;
processing the data to be written through the target check disk, and determining a first check value corresponding to the data to be written;
and writing the first check value into the target check disk through the target check disk.
3. The method of claim 1, wherein the writing the data to be written to the target data disk via the target data disk comprises:
determining, by the target data disk, if there is first data in the plurality of data disks, whether there is second data in the target data disk;
under the condition that the target data disk does not have the second data, writing the data to be written into the target data disk through the target data disk;
updating a check value in the target check disk based on the data to be written through the target check disk; and the check value in the target check disk is the check value corresponding to the first data.
4. The method of claim 3, wherein the updating, by the target parity disk, the parity value in the target parity disk based on the data to be written comprises:
and updating the second check value based on the data to be written and the second check value in the target check disk through the target check disk.
5. The method of claim 4, wherein updating, by the target parity disk, the second parity value based on the data to be written and the second parity value in the target parity disk comprises:
determining a third check value based on the data to be written and a second check value in the target check disk through the target check disk;
and replacing the second check value in the target check disk based on the third check value through the target check disk.
6. The method of claim 1, wherein updating, by the target check disk, the check value in the target check disk based on the delta data comprises:
and updating a fourth check value based on the incremental data and the fourth check value in the target check disk through the target check disk.
7. The method of claim 6, wherein updating, by the target check disk, the fourth check value based on the delta data and the fourth check value in the target check disk comprises:
determining a fifth check value based on the incremental data and a fourth check value in the target check disk through the target check disk;
and replacing, by the target check disk, a fourth check value in the target check disk based on the fifth check value.
8. The method of claim 1, wherein the obtaining data write information comprises:
receiving a data writing request sent by first electronic equipment, and analyzing the data writing request to obtain data writing information;
correspondingly, the method further comprises the following steps:
monitoring whether the data to be written is successfully written into the target data disk or not through the target data disk;
generating feedback information based on a monitoring result and sending the feedback information to the first electronic equipment through the target data disk; wherein the feedback information is used to instruct the first electronic device to determine whether to reinitiate the data write request.
9. A second electronic device, characterized in that the second electronic device comprises: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is used for executing the data processing program stored in the memory to realize the following steps:
acquiring data writing information; the data writing information comprises data to be written and position information corresponding to the data to be written; the position information represents the writing position of the data to be written;
determining a target data disk from a plurality of data disks based on the position information and the size of the data to be written;
writing the data to be written into the target data disk through the target data disk;
wherein writing the data to be written to the target data disk by the target data disk includes:
under the condition that the plurality of data disks have first data and the target data disk has second data, writing the data to be written into the target data disk through the target data disk, and determining incremental data based on the data to be written and the second data;
and updating the check value in the target check disk based on the incremental data through the target check disk.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs which are executable by one or more processors to implement the steps of the data processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322457.4A CN113031869B (en) | 2021-03-25 | 2021-03-25 | Data processing method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322457.4A CN113031869B (en) | 2021-03-25 | 2021-03-25 | Data processing method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113031869A CN113031869A (en) | 2021-06-25 |
| CN113031869B true CN113031869B (en) | 2023-02-03 |
Family
ID=76473984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110322457.4A Active CN113031869B (en) | 2021-03-25 | 2021-03-25 | Data processing method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113031869B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976175A (en) * | 2010-08-19 | 2011-02-16 | 北京同有飞骥科技有限公司 | Method for constructing disk array by horizontal grouping parallel concentrated verification |
| CN102981927A (en) * | 2011-09-06 | 2013-03-20 | 阿里巴巴集团控股有限公司 | Distribution type independent redundant disk array storage method and distribution type cluster storage system |
| CN106293990A (en) * | 2016-08-10 | 2017-01-04 | 华中科技大学 | A kind of RAID method based on batch write check |
| CN110618895A (en) * | 2019-09-29 | 2019-12-27 | 北京天融信网络安全技术有限公司 | Data updating method and device based on erasure codes and storage medium |
| CN111506251A (en) * | 2019-01-30 | 2020-08-07 | 杭州海康威视系统技术有限公司 | Data processing method, data processing device, SMR storage system and storage medium |
| CN112286725A (en) * | 2020-10-28 | 2021-01-29 | 联想(北京)有限公司 | Data processing method and device and electronic equipment |
-
2021
- 2021-03-25 CN CN202110322457.4A patent/CN113031869B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976175A (en) * | 2010-08-19 | 2011-02-16 | 北京同有飞骥科技有限公司 | Method for constructing disk array by horizontal grouping parallel concentrated verification |
| CN102981927A (en) * | 2011-09-06 | 2013-03-20 | 阿里巴巴集团控股有限公司 | Distribution type independent redundant disk array storage method and distribution type cluster storage system |
| CN106293990A (en) * | 2016-08-10 | 2017-01-04 | 华中科技大学 | A kind of RAID method based on batch write check |
| CN111506251A (en) * | 2019-01-30 | 2020-08-07 | 杭州海康威视系统技术有限公司 | Data processing method, data processing device, SMR storage system and storage medium |
| CN110618895A (en) * | 2019-09-29 | 2019-12-27 | 北京天融信网络安全技术有限公司 | Data updating method and device based on erasure codes and storage medium |
| CN112286725A (en) * | 2020-10-28 | 2021-01-29 | 联想(北京)有限公司 | Data processing method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113031869A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1533695B1 (en) | Updating data in a mobile terminal | |
| CN110178114B (en) | Vehicle control device and program update system | |
| CN107783776B (en) | Processing method and device of firmware upgrade package and electronic equipment | |
| CN111813428A (en) | Method and device for upgrading terminal firmware, electronic equipment and storage medium | |
| CN108762787B (en) | Software repairing method, device, computer equipment and storage medium | |
| CN108196975B (en) | Data verification method and device based on multiple checksums and storage medium | |
| CN113031869B (en) | Data processing method and device and computer readable storage medium | |
| CN109766084B (en) | Customized development method and device for payment application, computer equipment and storage medium | |
| CN112882743A (en) | Software upgrading method | |
| CN115344434B (en) | Patching method and device, electronic equipment and storage medium | |
| CN113505026B (en) | Method, device and equipment for verifying backup data and storage medium | |
| CN108762801B (en) | Software repair package processing method, device, computer equipment and storage medium | |
| CN114138303A (en) | Vehicle application program updating method, device, medium and vehicle | |
| CN114915554A (en) | Remote upgrading method and device, computer equipment and storage medium | |
| CN108133149B (en) | Data protection method and device and electronic equipment | |
| CN111124932A (en) | Scheme verification method, system, device, computer equipment and storage medium | |
| CN110825406A (en) | Software upgrading method and related equipment | |
| US11886869B2 (en) | Information processing device and information processing method for updating a program by partially relocating the program | |
| CN107656747B (en) | Data upgrading method and mobile terminal | |
| CN112860284B (en) | SP upgrade package generation method and device for equipment remote upgrade and computer equipment | |
| CN112165406B (en) | Interface message testing method and device, computer equipment and storage medium | |
| CN115687074A (en) | Business system testing method and device, computer equipment and storage medium | |
| KR20080065166A (en) | Application program download apparatus and method for touch panel in mobile terminal | |
| CN115658169A (en) | Controller starting method, device, equipment and medium | |
| CN113419756A (en) | File upgrading method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |