CN111708485A - Method for reading and writing data in storage medium - Google Patents
Method for reading and writing data in storage medium Download PDFInfo
- Publication number
- CN111708485A CN111708485A CN202010445395.1A CN202010445395A CN111708485A CN 111708485 A CN111708485 A CN 111708485A CN 202010445395 A CN202010445395 A CN 202010445395A CN 111708485 A CN111708485 A CN 111708485A
- Authority
- CN
- China
- Prior art keywords
- data
- information table
- storage medium
- position information
- reading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000010276 construction Methods 0.000 claims abstract description 15
- 230000004044 response Effects 0.000 claims abstract description 3
- 238000000547 structure data Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000013499 data model Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a method for reading data from a storage medium, which comprises the following steps: judging whether target data in the storage medium is sparse data or not, and determining a read head bit and a read tail bit according to a position information table in response to the fact that the target data is sparse data; reading the target data in the storage medium for one time based on the first reading bit and the last reading bit to obtain construction data, and obtaining a construction position information table based on the position information table; and intercepting all effective data in the constructed data according to the constructed position information table, and merging all effective data to obtain the fully effective data. The invention also discloses a method for writing data by the storage medium. According to the invention, multiple times of reading and writing of the sparse data of the storage medium are combined into one time of reading, and multiple times of writing are combined into one time of writing, so that the reading and writing times of the storage medium are greatly reduced, the efficiency of reconstructing the stored data is improved, and the reconstruction performance is improved.
Description
Technical Field
The invention relates to the technical field of storage medium reading and writing, in particular to a method for reading and writing data of a storage medium.
Background
In the distributed file system, storage media need to be read and written, such as data storage, data replication, data balancing, data recovery and the like. For a random small IO data model, the data storage mode is not continuous, and the data is sparsely stored on a continuous data block, that is, the data is dispersed in segments, and the middle is a hole. For example, 8K data is randomly written for the first time, 8K data is randomly written for the second time, the two 8K data are distributed at two positions of the data block, and the middle and the head and the tail are both holes. The data blocks described above are abstracted into objects. Data reconstruction is performed in a minimum unit of an object. When the object data is read, the object data is read section by section, holes are skipped, and finally the object data is merged and sent to other storage equipment. Other storage devices also receive writes in segments. Taking 100 pieces of data in an object as an example, 100 times will be read when reading data, and 100 times will be written when writing data. For mechanical hard disks, this approach tends to be inefficient. For solid state disk, although the random read-write speed is much faster than that of mechanical disk, the read-write 100 times wastes much unnecessary time.
Disclosure of Invention
In view of this, an object of the embodiments of the present invention is to provide a method for reading and writing data in a storage medium, in which multiple times of reading and writing sparse data in the storage medium are combined into one time of reading, and multiple times of writing are combined into one time of writing, so as to greatly reduce the number of times of reading and writing the storage medium, improve the efficiency of reconstructing stored data, and improve the reconstruction performance.
In view of the above object, an aspect of the embodiments of the present invention provides a method for reading data from a storage medium, including the following steps: judging whether target data in the storage medium is sparse data or not, and determining a read head bit and a read tail bit according to a position information table in response to the fact that the target data is sparse data; reading the target data in the storage medium for one time based on the first reading bit and the last reading bit to obtain construction data, and obtaining a construction position information table based on the position information table; and intercepting all effective data in the constructed data according to the constructed position information table, and merging all effective data to obtain the fully effective data.
In some embodiments, determining the read head bit and the read tail bit from the location information table includes: obtaining the offset of the initial effective data and the offset and the length of the termination effective data according to the position information table; the offset of the start valid data is set to the read head bit and the offset of the end valid data added to the length is set to the read tail bit.
In some embodiments, reading the target data in the storage medium once based on the read first bit and the read last bit to obtain the configuration data comprises: and sequentially reading target data in the storage medium from the first bit to the last bit at one time, wherein the target data comprises valid data and/or hole data.
In some embodiments, deriving the constructed location information table based on the location information table comprises: subtracting the offsets of a plurality of effective data in the position information table from the reading head bits respectively to obtain corresponding constructed data offsets; and replacing the corresponding effective data offset in the position information table by the constructed data offset to obtain a constructed position information table.
In some embodiments, further comprising: and encapsulating the full valid data and the position information table into messages, and sending the messages to other storage media to write data.
In another aspect of the embodiments of the present invention, a method for writing data in a storage medium is further provided, including the following steps: obtaining a constructed position information table based on the position information table, and carrying out zero filling splicing on the full effective data according to the constructed position information table to obtain constructed data; determining a writing head according to the position information table; and writing the construction data to the storage medium at a time based on the head bit.
In some embodiments, deriving the constructed location information table based on the location information table comprises: subtracting the offsets of a plurality of effective data in the position information table from the reading head bits respectively to obtain corresponding constructed data offsets; and replacing the corresponding effective data offset in the position information table by the constructed data offset to obtain a constructed position information table.
In some embodiments, zero padding and splicing the fully valid data according to the construction position information table to obtain the construction data comprises: obtaining the offset and the length of a plurality of effective data according to the constructed position information table; and sequentially reading the fully-valid data based on the length of the plurality of valid data, and carrying out zero padding based on the offset of the plurality of valid data.
In some embodiments, determining the head of write from the table of location information comprises: obtaining the offset of the initial effective data according to the position information table; the offset of the start valid data is set to the head of write.
In some embodiments, writing the configuration data to the storage medium at a time based on the header bit comprises: the structured data is written to the storage medium sequentially at once, starting from the corresponding first write bit in the storage medium.
The invention has the following beneficial technical effects: the method has the advantages that multiple times of reading and writing of the sparse data of the storage medium are combined into one time of reading, and multiple times of writing are combined into one time of writing, so that the reading and writing times of the storage medium are greatly reduced, the efficiency of reconstructing the stored data is improved, and the reconstruction performance is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a schematic diagram of an embodiment of a method for reading data from a storage medium provided by the present invention;
FIG. 2 is a process diagram of an embodiment of a method of reading data from a storage medium provided by the present invention;
FIG. 3 is a diagram illustrating an embodiment of a method for writing data on a storage medium according to the present invention;
fig. 4 is a process diagram of an embodiment of a method for writing data on a storage medium provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the above objects, a first aspect of embodiments of the present invention proposes an embodiment of a method for reading data from a storage medium. FIG. 1 is a schematic diagram illustrating an embodiment of a method for reading data from a storage medium according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s01, judging whether the target data in the storage medium is sparse data, responding to the fact that the target data is sparse data, and determining a read head and a read tail according to the position information table;
s02, reading the target data in the storage medium once based on the first reading bit and the last reading bit to obtain the constructed data, and obtaining a constructed position information table based on the position information table; and
and S03, intercepting all effective data in the constructed data according to the constructed position information table, and merging all effective data to obtain the fully effective data.
In this embodiment, an Object-based storage device OSD (Object-based storage device) in the distributed file system mainly performs operations of storing data, copying data, balancing data, restoring data, and the like, and generally, one hard disk corresponds to one OSD, and the OSD manages the hard disk storage. When the hard disk mounted by the OSD fails, the data of the OSD needs to be recovered, and the process of recovering the data in the failed OSD from other OSD is called data reconstruction; when the distributed file system is expanded, the OSD performs data balance, and a part of data is migrated to the OSD of a new expansion node, which is also called data reconstruction. When data reconstruction occurs, the client performs read-write operation and data reconstruction on the cluster at the same time. When the client reads and writes data, the client finally reads and writes a specific storage medium (such as a hard disk) through the main OSD; when reconstructing data, it is also necessary to read from or write to a specific storage medium (e.g., a hard disk). For a random small IO data model, the storage mode of data on the hard disk is not continuous, and the data is stored sparsely on a continuous data block, that is, the data is a section of data, and the middle is a hole. For example, 8K data is randomly written for the first time, 8K data is randomly written for the second time, the two 8K data are distributed at two positions of the data block, and the middle and the head and the tail are both holes. Data reconstruction is performed in a minimum unit of an object. Taking the data of the standby OSD restored by the main OSD as an example, the main OSD determines which objects need to be restored by the standby OSD and then needs to be restored one by one. Taking the example of recovering an object, the main OSD reads the data of the object from its own hard disk, then encapsulates the data into a message and sends the message to the standby OSD, and the standby OSD writes the data into its own hard disk after receiving the message.
In this embodiment, after the main OSD determines that the mode target data is sparse data, the first bit and the last bit are determined according to the position information table, the target data in the hard disk is read once, the read data includes all valid data and does not skip a hole, the configuration data is obtained, the configuration position information table is obtained according to the position information table, the configuration data is intercepted and merged to obtain fully valid data, and finally the fully valid data and the position information table are packaged into a message to be sent to the standby OSD.
FIG. 2 is a process diagram illustrating an embodiment of a method for reading data from a storage medium provided by the present invention. As shown in fig. 2, in this embodiment, (a) is target data in the hard disk, four white areas are hole data, three gray areas are valid data, and the position information table is: { [1048576, 4096], [1887436, 4096], [3145728, 4096] }, wherein [1048576, 4096] denotes an offset of the start valid data of 1048576; the data length of the start valid data is 4096. The first bit is the offset 1048576 for the start valid data and the last bit is the offset 3145728 for the end valid data plus 3149824 for the length 4096.
Reading target data in the storage medium sequentially from a first reading bit 1048576 to a last reading bit 3149824 in (a), wherein the reading of the data of the unique hole is not ended continuously, the structural data (b) with the total length of 3145728+ 4096-: {[0, 4096],[838860, 4096],[2097152, 4096]}.
Traversing and constructing a position information table, intercepting and combining three sections of effective data from the array buffer to obtain fully effective data (c), packaging the fully effective data (c) and the position information table into messages, and sending the messages to other storage media to write data.
In some embodiments of the present invention, determining the read head bit and the read tail bit from the location information table comprises: obtaining an offset (e.g. 1048576) of the start valid data, an offset (e.g. 3145728) and a length (e.g. 4096) of the end valid data according to the position information table; the offset of the start valid data is set to the read head bit (e.g., 1048576) and the offset of the end valid data (e.g., 3145728) is added to the length (e.g., 4096) (e.g., 3149824) to the read tail bit.
In some embodiments of the present invention, reading the target data in the storage medium once based on the read leading bit and the read trailing bit to obtain the configuration data comprises: and sequentially reading target data in the storage medium from the first bit to the last bit at one time, wherein the target data comprises valid data and/or hole data.
In some embodiments of the invention, deriving the constructed location information table based on the location information table comprises: subtracting offsets of a plurality of effective data in a position information table, such as { [1048576, 4096], [1887436, 4096], [3145728, 4096] }, from offsets (such as 1048576) of the initial effective data respectively to obtain corresponding constructed data offsets; and replacing the corresponding effective data offset in the position information table by the construction data offset to obtain a construction position information table, such as { [0, 4096], [838860, 4096], [2097152, 4096 }.
In some embodiments of the invention, further comprising: and encapsulating the full valid data and the position information table into messages, and sending the messages to other storage media to write data.
It should be particularly noted that, the steps in the embodiments of the method for reading data from a storage medium can be mutually intersected, replaced, added, or deleted, and therefore, these methods for reading data from a storage medium by reasonable permutation and combination conversion also belong to the scope of the present invention, and the scope of the present invention should not be limited to the embodiments.
In view of the above objects, in another aspect of the embodiments of the present invention, an embodiment of a method for writing data by a storage medium is provided. Fig. 3 is a schematic diagram illustrating an embodiment of a method for writing data on a storage medium according to the present invention. As shown in fig. 3, the embodiment of the present invention includes the following steps:
s11, obtaining a constructed position information table based on the position information table, and carrying out zero filling splicing on the full effective data according to the constructed position information table to obtain constructed data;
s12, determining a write head according to the position information table; and
and S13, writing the structure data into the storage medium at one time based on the head writing bit.
In this embodiment, the standby OSD receives a message encapsulated by the full valid data and the position information table, obtains a constructed position information table based on the position information table, performs zero padding and splicing on the full valid data according to the constructed position information table to obtain constructed data, determines a write head according to the position information table, and sequentially writes the constructed data into the standby OSD at one time.
Fig. 4 is a process diagram illustrating an embodiment of a method for writing data on a storage medium provided by the present invention. As shown in fig. 4, in the present embodiment, the offset of valid data in the position information table is subtracted from the offset 1048576 of the start valid data to obtain a structural position information table: {[0, 4096],[838860, 4096],[2097152, 4096]}. And performing zero padding and splicing on the fully valid data (c) according to the constructed position information table, reading 4096 bits of the fully valid data (c) according to the offset and the length [0, 4096] of the first valid data in the constructed position information table to obtain a data segment (d), performing zero padding on the valid data (d) according to the offset and the length [838860, 4096] of the second valid data to obtain 830668 bits, reading 4096 bits of the second valid data (c) to obtain a data segment (e), and similarly performing zero padding and splicing according to the offset and the length [2097152, 4096] of the third valid data to obtain the constructed data (f). Based on the start valid data offset 1048576 in the position information table, the writing of the construction data (f) into the hard disk is started.
In some embodiments of the invention, constructing the table of location information based on the table of location information comprises: subtracting offsets of a plurality of effective data in a position information table, such as { [1048576, 4096], [1887436, 4096], [3145728, 4096] }, from offsets (such as 1048576) of the initial effective data respectively to obtain corresponding constructed data offsets; and replacing the corresponding effective data offset in the position information table by the construction data offset to obtain a construction position information table, such as { [0, 4096], [838860, 4096], [2097152, 4096 }.
In some embodiments of the present invention, zero padding and splicing the fully valid data according to the configuration location information table to obtain the configuration data comprises: obtaining the offset and the length of a plurality of effective data according to the constructed position information table; and sequentially reading the fully-valid data based on the length of the plurality of valid data, and carrying out zero padding based on the offset of the plurality of valid data.
In some embodiments of the present invention, determining the head of write from the location information table comprises: obtaining an offset 1048576 of the initial valid data according to the position information table; the offset of the start valid data is set to the header bit 1048576.
In some embodiments of the present invention, writing the configuration data to the storage medium at once based on the head of write bit 1048576 comprises: the structured data is written sequentially to the storage medium at a time, starting with the corresponding write head 1048576 in the storage medium.
It should be particularly noted that, the steps in the embodiments of the method for writing data on a storage medium described above can be mutually intersected, replaced, added, or deleted, and therefore, these methods for writing data on a storage medium, which are transformed by reasonable permutation and combination, should also belong to the scope of the present invention, and should not limit the scope of the present invention to the embodiments.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate, all or part of the processes in the methods of the above embodiments may be implemented by instructing relevant hardware by a computer program, and the program of the method for reading and writing data in a storage medium may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
The steps of a method or algorithm described in connection with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, storage medium, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, storage media storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, storage media and optical disks include Compact Disks (CDs), laser disks, optical disks, Digital Versatile Disks (DVDs), floppy disks, blu-ray disks where storage media usually reproduce data magnetically, while optical disks reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, such as a read-only memory, a storage medium, or an optical disc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method of reading data from a storage medium, comprising the steps of:
judging whether target data in a storage medium is sparse data or not, and determining a read head bit and a read tail bit according to a position information table in response to the fact that the target data is sparse data;
reading the target data in the storage medium once based on the read head bit and the read tail bit to obtain construction data, and obtaining a construction position information table based on the position information table; and
and intercepting all effective data in the constructed data according to the constructed position information table, and merging all effective data to obtain fully effective data.
2. The method of reading data from a storage medium according to claim 1, wherein determining a read head bit and a read tail bit from the location information table comprises:
obtaining the offset of the initial effective data and the offset and the length of the termination effective data according to the position information table;
and setting the offset of the start effective data as a read head bit, and setting the offset of the end effective data and the sum of the length as a read tail bit.
3. The method of reading data from a storage medium according to claim 1, wherein reading the target data from the storage medium once based on the read head bit and the read tail bit to obtain the configuration data comprises:
and sequentially reading the target data in the storage medium from the read head bit to the read tail bit at one time, wherein the target data comprises valid data and/or hole data.
4. The method of reading data from a storage medium according to claim 1, wherein obtaining a constructed location information table based on the location information table comprises:
subtracting the offsets of a plurality of effective data in the position information table from the reading head bits respectively to obtain corresponding constructed data offsets;
and replacing the corresponding effective data offset in the position information table by the constructed data offset to obtain a constructed position information table.
5. The method of reading data from a storage medium of claim 1, further comprising:
and encapsulating the full valid data and the position information table into messages, and sending the messages to other storage media to write data.
6. A method for writing data on a storage medium, comprising the steps of:
obtaining a constructed position information table based on a position information table, and carrying out zero padding splicing on the full effective data according to the constructed position information table to obtain constructed data;
determining a writing head according to the position information table; and
writing the constructed data to a storage medium at a time based on the write header.
7. The method for writing data on a storage medium according to claim 6, wherein obtaining a configuration location information table based on the location information table comprises:
subtracting the offsets of a plurality of effective data in the position information table from the reading head bits respectively to obtain corresponding constructed data offsets;
and replacing the corresponding effective data offset in the position information table by the constructed data offset to obtain a constructed position information table.
8. The method for writing data to the storage medium according to claim 6, wherein zero padding and splicing the fully valid data according to the structure location information table to obtain the structure data comprises:
obtaining the offset and the length of a plurality of effective data according to the constructed position information table;
sequentially reading the fully-valid data based on the length sequence of the plurality of valid data, and performing zero padding based on the offset of the plurality of valid data.
9. The method for writing data on a storage medium according to claim 6, wherein determining the head bit according to the location information table comprises:
obtaining the offset of the initial effective data according to the position information table;
and setting the offset of the initial valid data as a head-of-write bit.
10. The method of writing data to a storage medium of claim 6, wherein writing the structured data to the storage medium at a time based on the write header comprises:
and sequentially writing the construction data into the storage medium at one time from the corresponding write head in the storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010445395.1A CN111708485A (en) | 2020-05-24 | 2020-05-24 | Method for reading and writing data in storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010445395.1A CN111708485A (en) | 2020-05-24 | 2020-05-24 | Method for reading and writing data in storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111708485A true CN111708485A (en) | 2020-09-25 |
Family
ID=72537434
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010445395.1A Pending CN111708485A (en) | 2020-05-24 | 2020-05-24 | Method for reading and writing data in storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708485A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1737745A (en) * | 2004-08-18 | 2006-02-22 | 华为技术有限公司 | Read-write method for disc array data and parallel read-write method |
| CN105373487A (en) * | 2015-12-16 | 2016-03-02 | 浪潮(北京)电子信息产业有限公司 | Defragmentation method and system for storage operation system |
| CN110740138A (en) * | 2019-10-24 | 2020-01-31 | 北京百度网讯科技有限公司 | Data transmission method and device |
| CN110888843A (en) * | 2019-10-31 | 2020-03-17 | 北京浪潮数据技术有限公司 | Cross-host sparse file copying method, device, equipment and storage medium |
| US20200097205A1 (en) * | 2018-09-24 | 2020-03-26 | Salesforce.Com, Inc. | System and method for early removal of tombstone records in database |
| CN111104063A (en) * | 2019-12-06 | 2020-05-05 | 浪潮电子信息产业股份有限公司 | Data storage method and device, electronic equipment and storage medium |
-
2020
- 2020-05-24 CN CN202010445395.1A patent/CN111708485A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1737745A (en) * | 2004-08-18 | 2006-02-22 | 华为技术有限公司 | Read-write method for disc array data and parallel read-write method |
| CN105373487A (en) * | 2015-12-16 | 2016-03-02 | 浪潮(北京)电子信息产业有限公司 | Defragmentation method and system for storage operation system |
| US20200097205A1 (en) * | 2018-09-24 | 2020-03-26 | Salesforce.Com, Inc. | System and method for early removal of tombstone records in database |
| CN110740138A (en) * | 2019-10-24 | 2020-01-31 | 北京百度网讯科技有限公司 | Data transmission method and device |
| CN110888843A (en) * | 2019-10-31 | 2020-03-17 | 北京浪潮数据技术有限公司 | Cross-host sparse file copying method, device, equipment and storage medium |
| CN111104063A (en) * | 2019-12-06 | 2020-05-05 | 浪潮电子信息产业股份有限公司 | Data storage method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11307776B2 (en) | Method for accessing distributed storage system, related apparatus, and related system | |
| CN111177043B (en) | Method, system, device and medium for accelerating reading of field replaceable unit information | |
| CN110519329B (en) | Method, device and readable medium for concurrently processing samba protocol request | |
| CN111611109A (en) | Backup method, system, device and medium for distributed cluster | |
| CN111352586B (en) | Directory aggregation method, device, equipment and medium for accelerating file reading and writing | |
| CN111240595A (en) | Method, system, equipment and medium for optimizing storage cache | |
| CN111104232A (en) | Method, device and medium for accelerating message writing of message queue | |
| CN111338852A (en) | Method, system, equipment and medium for creating storage double live volume snapshot | |
| CN111352589B (en) | Distributed storage method, device, equipment and readable medium | |
| CN112785408A (en) | Account checking method and device based on Hash | |
| CN111221826A (en) | Method, system, device and medium for processing shared cache synchronization message | |
| CN111625203A (en) | Method, system, device and medium for hierarchical storage | |
| CN115146002A (en) | Cross-data-center data synchronization method and device | |
| CN111857978A (en) | Distributed transaction processing method and device, computer equipment and storage medium | |
| CN111274616B (en) | Ceph object storage based integral uploading segmented verification realization method | |
| CN111061698B (en) | Method and device for storing Ethernet contract data | |
| CN110781164B (en) | Design method, device and medium of database all-in-one machine | |
| CN111708485A (en) | Method for reading and writing data in storage medium | |
| CN106469172B (en) | The data-updating method and device of distributed file system | |
| CN112202862A (en) | Kafka-based cluster data and file synchronization method and device | |
| CN111813346A (en) | Method, system, device and medium for establishing Ceph distributed storage based on cloud platform | |
| CN112988880A (en) | Data synchronization method and device, electronic equipment and computer storage medium | |
| CN110727534A (en) | Method, equipment and storage medium for repairing file system | |
| CN114253870A (en) | Method, system, device and medium for updating L2P table | |
| WO2022033269A1 (en) | Data processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200925 |