CN104598161B - Digital independent, wiring method and device and data store organisation - Google Patents

Digital independent, wiring method and device and data store organisation Download PDF

Info

Publication number
CN104598161B
CN104598161B CN201310534962.0A CN201310534962A CN104598161B CN 104598161 B CN104598161 B CN 104598161B CN 201310534962 A CN201310534962 A CN 201310534962A CN 104598161 B CN104598161 B CN 104598161B
Authority
CN
China
Prior art keywords
data block
data
hard disk
index
page
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201310534962.0A
Other languages
Chinese (zh)
Other versions
CN104598161A (en
Inventor
陈明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201310534962.0A priority Critical patent/CN104598161B/en
Publication of CN104598161A publication Critical patent/CN104598161A/en
Application granted granted Critical
Publication of CN104598161B publication Critical patent/CN104598161B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

A kind of digital independent of present invention offer, wiring method and device and data store organisation, including step:Obtain the address number of data block;According to address number and data directory, the index content in data directory is obtained;Page number and hard disk quantity are stored according to the data block overall situation, obtains the hard disk reel number and data block local storage page number of data block;Number of pages, hard disk reel number and data block local storage page number are stored according to data block, submits the asynchronous read requests of data block;And according to asynchronous read requests, after operating system read block, in readjustment to corresponding process flow;Wherein hard disk carries out paging processing with the physical page of hard disk, to form the memory page of the hard disk for the physical page for including at least one hard disk;Each memory page corresponds to a data block.Digital independent, wiring method and the device and data store organisation of the present invention can obtain the position of corresponding data block by data directory, and the only corresponding data block of each memory page can accelerate the reading and write operation of data block.

Description

Digital independent, wiring method and device and data store organisation
Technical field
The present invention relates to data processing fields, are deposited more particularly to a kind of digital independent, wiring method and device and data Storage structure.
Background technology
SSD (Solid State Disk, solid state disk) is a kind of storage device based on flash memory, is stored with solid-state electronic Chip array and manufactured hard disk, by control unit and storage unit (FLASH chip or DRAM (Dynamic Random Access Memory, dynamic random access memory) chip) composition.SSD is inserted into I/O buses in standard hard drive slot, place The request of read-write logic magnetic disc block of the reason from CPU (Central Processing Unit, central processing unit).One SSD packet One or more flash chips and flash memory translation layer are included, flash chip substitutes the thermo-mechanical drive in traditional magnetic disk, and flash memory turns over It is a hardware/firmware equipment to translate layer then, identical as the effect of Magnetic Disk Controler, will translate into the request of logical block to bottom The access of layers of physical devices.
As shown in table 1, the sequence read operation (CPU accesses logic magnetic disc block in order) of SSD disks and sequence write operation at present Performance it is suitable.But, when CPU is according to random-sequential access logic magnetic disc block, write operation is more an order of magnitude slower than read operation.
Table 1
The performance difference of random read operation and random writing operations is determined by bottom flash memory essential attribute.Such as a flash memory It is made of the sequence of m block, each block is formed by n pages.Data are written and read operation as unit of page, only belonging to one page Block by after entire wiped, write operation could be carried out in the page.
When carrying out random writing operations, erasing block needs the relatively long time (ms grades), when time needed for the read operation High an order of magnitude.It is all in this page to carry if write operation attempts to change the page of included data simultaneously The block of data all must be copied to another page, then could carry out write operation to the page.
Since the random read operation of current SSD disks and the operating time of random writing operations differ greatly, data is caused to be read It takes and easily blocks process flow when being written, the application process in flow can not effectively using the computing resource of equipment and deposit Store up resource.
Invention content
The embodiment of the present invention is designed to provide a kind of method for reading data, is held with solving existing method for reading data Easily obstruction application process, causes application process not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
The purpose of the embodiment of the present invention, which also resides in, provides a kind of method for writing data, to solve existing method for writing data It is easy obstruction application process, application process is caused not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
The purpose of the embodiment of the present invention, which also resides in, provides a kind of data store organisation, to solve existing digital independent and write Enter method and be easy obstruction application process, causes application process that can not effectively utilize the computing resource and storage resource of equipment Technical problem.
The purpose of the embodiment of the present invention, which also resides in, provides a kind of digital independent device, to solve existing digital independent device It is easy obstruction application process, application process is caused not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
The purpose of the embodiment of the present invention, which also resides in, provides a kind of data transfer apparatus, to solve existing data transfer apparatus It is easy obstruction application process, application process is caused not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
To solve the above problems, technical solution provided by the invention is as follows:
There is provided a kind of method for reading data comprising step:
Obtain the address number of data block;
According to described address number and data directory, the index content in the data directory is obtained, wherein in the index Appearance includes data block storage number of pages and data block overall situation storage page number;
Page number and hard disk quantity are stored according to the data block overall situation, obtains the hard disk reel number and number of the data block According to block local storage page number;
Number of pages, the hard disk reel number and the data block local storage page number are stored according to the data block, submits number According to the asynchronous read requests of block;And
After operating system read block, according to the asynchronous read requests, adjust back into corresponding process flow;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one described hard to be formed The memory page of the hard disk of the physical page of disk;Each memory page corresponds to a data block;According to the number The data block overall situation is generated according to the memory page of block and stores page number, and generating the data block according to the size of the data block deposits Store up number of pages.
Also provide a kind of method for writing data comprising step:
According to data directory, the data block overall situation storage page number of the memory page of the blank of the hard disk is obtained;
Page number is stored according to the data block overall situation of the data block and the memory page of the blank, is generated and the number According to the corresponding index content of block and address number, deposited wherein the index content includes data block storage number of pages and the data block overall situation Store up page number;
Page number and hard disk quantity are stored according to the data block overall situation, generates the hard disk reel number and number of the data block According to block local storage page number;
Number of pages, the hard disk reel number and the data block local storage page number are stored according to the data block, submits number According to block write request;And
According to said write request as a result, updating the data directory using the index content;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one described hard to be formed The memory page of the hard disk of the physical page of disk;Each memory page corresponds to a data block;According to the number The data block overall situation is generated according to the memory page of block and stores page number, and generating the data block according to the size of the data block deposits Store up number of pages.
A kind of data store organisation is also provided comprising:
Data store, including multiple hard disks, the hard disk carries out paging processing with the physical page of the hard disk, with shape At the memory page of the hard disk of the physical page including at least one hard disk;Wherein each memory page corresponds to One data block;
Data directory portion, including index content, including:
Data block stores number of pages, the size for indicating the data block;And
The data block overall situation stores page number, the storage location for indicating the data block;And
Address number, it is corresponding with the index content, for being supplied to user, with read block.
A kind of digital independent device is also provided comprising:
Address acquisition module, the address number for obtaining data block;
Acquisition module is indexed, for according to described address number and data directory, obtaining in the index in the data directory Hold, wherein the index content includes data block storage number of pages and data block overall situation storage page number;
Position acquisition module obtains the data for storing page number and hard disk quantity according to the data block overall situation The hard disk reel number and data block local storage page number of block;
Module is submitted in request, for storing number of pages, the hard disk reel number and the data block office according to the data block Portion stores page number, submits the asynchronous read requests of data block;And
Callback module, after operating system read block, for according to the asynchronous read requests, readjustment to corresponding place It manages in flow;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one described hard to be formed The memory page of the hard disk of the physical page of disk;Each memory page corresponds to a data block;According to the number The data block overall situation is generated according to the memory page of block and stores page number, and generating the data block according to the size of the data block deposits Store up number of pages.
A kind of data transfer apparatus is also provided comprising:
Position acquisition module, for according to data directory, the data block for obtaining the memory page of the blank of the hard disk to be complete Office's storage page number;
Generation module is indexed, for being stored according to the data block overall situation of the data block and the memory page of the blank Page number generates index content corresponding with the data block and address number, wherein the index content includes data block memory page Number and the data block overall situation store page number;
Position generation module generates the data for storing page number and hard disk quantity according to the data block overall situation The hard disk reel number and data block local storage page number of block;
Writing module is locally deposited for storing number of pages, the hard disk reel number and the data block according to the data block Page number is stored up, data block write request is submitted;And
Update module, for according to said write ask as a result, using the index content update data directory;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one described hard to be formed The memory page of the hard disk of the physical page of disk;Each memory page corresponds to a data block;According to the number The data block overall situation is generated according to the memory page of block and stores page number, and generating the data block according to the size of the data block deposits Store up number of pages.
Compared to the prior art, digital independent of the invention, wiring method and device and data store organisation can pass through number Obtain the position of corresponding data block according to index, and the only corresponding data block of each memory page can accelerate the reading of data block with And write operation, it solves existing digital independent, wiring method and device and is easy obstruction application process, lead to application process The technical issues of computing resource and storage resource of equipment can not effectively be utilized.
Description of the drawings
Fig. 1 is that the working environment structure of the electronic equipment where the digital independent device and data transfer apparatus of the present invention is shown It is intended to;
Fig. 2 is the structural schematic diagram of the preferred embodiment of the data store organisation of the present invention;
Fig. 3 is the structural schematic diagram of the preferred embodiment of the digital independent device of the present invention;
Fig. 4 is the flow chart of the preferred embodiment of the method for reading data of the present invention;
Fig. 5 is the structural schematic diagram of the preferred embodiment of the data transfer apparatus of the present invention;
Fig. 6 is the flow chart of the preferred embodiment of the method for writing data of the present invention;
Fig. 7 is the flow chart of the method for writing data of the present invention and the specific embodiment of device;
Fig. 8 is the flow chart of the method for reading data of the present invention and the specific embodiment of device.
Specific implementation mode
Schema is please referred to, wherein identical component symbol represents identical component, the principle of the present invention is to implement one It is illustrated in computing environment appropriate.The following description be based on illustrated by the specific embodiment of the invention, should not be by It is considered as the limitation present invention other specific embodiments not detailed herein.
In the following description, specific embodiments of the present invention will be referred to by the operation performed by one or multi-section computer The step of and symbol illustrate, unless otherwise stating clearly.Therefore, these steps and operation be will appreciate that, carried for several times wherein having Include by representing with the computer disposal list of the electronic signal of the data in a structuring pattern to be executed by computer Member is manipulated.At this manipulation transforms data or the position being maintained in the memory system of the computer, it can match again It sets or in addition changes the running of the computer in a manner familiar to those skilled in the art.The data knot that the data are maintained Structure is the provider location of the memory, has the specific feature defined in the data format.But the principle of the invention is with above-mentioned Word illustrates, is not represented as a kind of limitation, those skilled in the art will appreciate that plurality of step as described below and Operation also may be implemented in hardware.
" component ", " module ", " system ", " interface ", " process " etc. are usually intended to as used herein the term Refer to computer related entity:Hardware, the combination of hardware and software, software or software in execution.For example, component can be but not It is limited to run process on a processor, processor, object, executable application, thread, program and/or the computer executed. By diagram, both the application and the controller run on the controller can be component.One or more components can have It is in the process executed and/or thread, and component can be located on a computer and/or be distributed in two or more meters Between calculation machine.
Moreover, claimed theme may be implemented as generating software using standard program and/or engineering technology, consolidate The method, apparatus or manufacture of disclosed theme are realized in part, hardware or its arbitrary combination with control computer.It is used herein Term " manufacture " be intended to comprising can be from arbitrary computer readable device, carrier or the computer program of medium access.Certainly, Those skilled in the art will appreciate that many modifications can be carried out to the configuration, without departing from claimed theme range or Spirit.
Fig. 1 and the discussion below are provided to realizing digital independent device of the present invention and data transfer apparatus institute Electronic equipment working environment it is brief, summarize description.The working environment of Fig. 1 is only the one of working environment appropriate A example and suggestion is not intended to about the purposes of working environment or any restrictions of the range of function.Example electronic equipment 112 Including but not limited to personal computer, server computer, hand-held or laptop devices, mobile device (such as mobile phone, Personal digital assistant (PDA), media player etc.), multicomputer system, consumer electronic devices, minicomputer, large size Computer includes the distributed computing environment, etc. of above-mentioned arbitrary system or equipment.
Although not requiring, in the common background that " computer-readable instruction " is executed by one or more electronic equipments Lower description embodiment.Computer-readable instruction can be distributed and (be discussed below) via computer-readable medium.It is computer-readable Instruction can be implemented as program module, such as the function of execution particular task or realization particular abstract data type, object, application Programming interface (API), data structure etc..Typically, the function of the computer-readable instruction can be in various environment arbitrarily Combination or distribution.
Fig. 1 illustrates the electricity of one or more embodiments of method for reading data and method for writing data including the present invention The example of sub- equipment 112.In one configuration, electronic equipment 112 includes at least one processing unit 116 and memory 118.Root According to the exact configuration and type of electronic equipment, memory 118 can be volatibility (such as RAM), it is non-volatile (such as ROM, flash memory etc.) or the two certain combination.The configuration is illustrated by dotted line 114 in Fig. 1.
In other embodiments, electronic equipment 112 may include supplementary features and/or function.For example, equipment 112 may be used also To include additional storage device (such as can be removed and/or non-removable) comprising but be not limited to magnetic memory apparatus, light is deposited Storage device etc..This additional memory devices are illustrated by storage device 120 in Fig. 1.In one embodiment, for realizing this The computer-readable instruction for one or more embodiments that text is provided can be in storage device 120.Storage device 120 may be used also To store other computer-readable instructions for realizing operating system, application program etc..Computer-readable instruction can be loaded into It is executed by such as processing unit 116 in memory 118.
Term as used herein " computer-readable medium " includes computer storage media.Computer storage media includes The volatibility that any method or technique of the information of such as computer-readable instruction or other data etc is realized for storage With non-volatile, removable and nonremovable medium.Memory 118 and storage device 120 are the examples of computer storage media. Computer storage media includes but not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital universal Disk (DVD) or other light storage devices, cassette tape, tape, disk storage device or other magnetic storage apparatus can be used for Any other medium that storage it is expected information and can be accessed by electronic equipment 112.Arbitrary such computer storage media can To be a part for electronic equipment 112.
Electronic equipment 112 can also include the communication connection 126 for allowing electronic equipment 112 to be communicated with other equipment.Communication Connection 126 can include but is not limited to modem, network interface card (NIC), integrated network interface, radiofrequency launcher/connect Receive device, infrared port, USB connections or other interfaces for electronic equipment 112 to be connected to other electronic equipments.Communication connection 126 may include wired connection or wireless connection.Communication connection 126 can emit and/or receive communication medium.
Term " computer-readable medium " may include communication media.Communication media typically comprises computer-readable instruction Or other data in " the own modulated data signal " of such as carrier wave or other transmission mechanisms etc, and include that any information is passed Send medium.Term " own modulated data signal " may include such signal:One or more of the characteristics of signals is according to general Mode in information coding to signal is set or changed.
Electronic equipment 112 may include input equipment 124, such as keyboard, mouse, pen, voice-input device, touch input Equipment, infrared camera, video input apparatus and/or any other input equipment.It can also include output equipment in equipment 112 122, such as one or more displays, loud speaker, printer and/or other arbitrary output equipments.Input equipment 124 and output Equipment 122 can be connected to electronic equipment 112 via wired connection, wireless connection or its arbitrary combination.In one embodiment, Input equipment or output equipment from another electronic equipment are used as the input equipment 124 or defeated of electronic equipment 112 Go out equipment 122.
The component of electronic equipment 112 can be connected by various interconnection (such as bus).Such interconnection may include outer Enclose component interconnection (PCI) (such as quick PCI), universal serial bus (USB), firewire (IEEE1394), optical bus structure etc. Deng.In another embodiment, the component of electronic equipment 112 can pass through network interconnection.For example, memory 118 can be by position Multiple physical memory cells arcs composition in different physical locations, by network interconnection.
It would be recognized by those skilled in the art that the storage device for storing computer-readable instruction can be across network point Cloth.For example, can be stored for realizing one provided by the present invention or more via the electronic equipment 130 that network 128 accesses The computer-readable instruction of a embodiment.Electronic equipment 112 can access electronic equipment 130 and downloading computer readable instruction It is part or all of for execute.Alternatively, electronic equipment 112 can download a plurality of computer-readable instruction on demand, Or some instructions can be executed at electronic equipment 112 and some instructions can execute at electronic equipment 130.
There is provided herein the various operations of embodiment.In one embodiment, one or more operations can be with structure At the computer-readable instruction stored on one or more computer-readable mediums, will make to succeed in one's scheme when being executed by electronic equipment It calculates equipment and executes the operation.Describing the sequences of some or all of operations, to should not be construed as to imply that these operations necessarily suitable Sequence is relevant.It will be appreciated by those skilled in the art that the alternative sequence of the benefit with this specification.Furthermore, it is to be understood that Not all operation must exist in each embodiment provided in this article.
Moreover, word " preferred " used herein means serving as example, example or illustration.Feng Wen is described as " preferably " any aspect or design be not necessarily to be construed as than other aspect or design it is more advantageous.On the contrary, the use purport of word " preferred " Concept is being proposed in specific ways.Term "or" as used in this application is intended to mean the "or" for including and non-excluded "or".That is, unless specified otherwise or clear from the context, " X uses A or B " means that nature includes any one of arrangement. That is, if X uses A;X uses B;Or X uses A and B both, then " X uses A or B " is met in aforementioned any example.
Moreover, although the disclosure, this field skill has shown and described relative to one or more realization methods Art personnel will be appreciated that equivalent variations and modification based on the reading and understanding to the specification and drawings.The disclosure include it is all this The modifications and variations of sample, and be limited only by the scope of the following claims.In particular, to by said modules (such as element, Resource etc.) various functions that execute, the term for describing such component is intended to correspond to the specified work(for executing the component The random component (unless otherwise instructed) of energy (such as it is functionally of equal value), even if illustrated herein with execution in structure The disclosure exemplary implementations in function open structure it is not equivalent.In addition, although the special characteristic of the disclosure Through being disclosed relative to the only one in several realization methods, but this feature can with such as can be to given or specific application For be it is expected and one or more other features combinations of other advantageous realization methods.Moreover, with regard to term " comprising ", " tool Have ", " containing " or its deformation be used in specific implementation mode or claim for, such term be intended to with term The similar mode of "comprising" includes.
Fig. 2 is please referred to, Fig. 2 is the structural schematic diagram of the preferred embodiment of the data store organisation of the present invention.This is preferred real The data store organisation 20 for applying example includes data store 21, data directory portion 22 and address number 23.Wherein data store 21 include multiple hard disks 211, which carries out paging processing with the physical page of hard disk 211, includes at least one to be formed The memory page 2111 of the hard disk 211 of the physical page of hard disk 211;Each memory page 2111 corresponds to a data block.It is i.e. each Hard disk 211 includes multiple memory pages 2111, and each memory page 2111 includes at least one physical page, physics same in this way The page can only store same data block.The physical page can just be carried out at erasing when therefore only deleting data block Reason need not carry out at copy and erasing the data block in the physical page to carry out the write operation of other data blocks Reason.
Data directory portion 22 includes index content 221 and index stem 222.Wherein index content 221 includes data block It stores number of pages 2211, data block overall situation storage page number 2212 and data and there is mark 2213.Data block stores number of pages 2211 and uses In the size for indicating data block;(data block is complete for storage location of the data block overall situation storage page number 2212 for indicating data block Office's storage page number 2212 has uniqueness in all data block overall situations storage page number of the hard disk of the electronic equipment);Data are deposited In mark 2213 for judging whether hard disc data corresponding with index content 221.It, can be fast by the index content 221 Validity, size information and the location information of the acquisition data block of speed.The index content 221 being arranged so simultaneously is not extra Information, corresponding memory space can be saved.
Index stem 222 includes use information, the use information of index and the shared label information of hard disk.Hard disk makes With information include hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The use of index Information includes used index quantity and not used index quantity;Shared label information is including magic number and uses mark Note.Wherein hard disk quantity indicates that the hard disk quantity of electronic equipment, hard disc physical number of pages indicate the physics of the hard disk 21 of electronic equipment The quantity of the page, hard disc physical page size indicate the size of the physical page of the hard disk 21 of electronic equipment, currently add Physical Page The page number of the not used physical page of the hard disk 21 of number expression electronic equipment.Magic number is used to indicate the number in the data directory portion According to validity (data in such as data directory portion are tampered, then the evil spirit number can change), using label for indicating number According to index use state (use state or unused state).By the index stem user can accurately obtain hard disk 21 with And the service condition and the data validity in data directory portion 22 and use state of index.
Address number 23 is for being supplied to user, to read corresponding data block.The rope of the address number 23 and respective data blocks It is corresponding to draw content 221.User can obtain corresponding index content 221 by address number 23 in this way, and then pass through index content 221 obtain corresponding data block.Preferably, in order to ensure the acquisition efficiency of index content, the address number 23 of data block and corresponding Index content 221 stored in the form of key-value (key-value).
Preferably, the stem for the data block that the data store organisation 20 of this preferred embodiment stores further includes the verification of data Page number and data actual length are stored with the data block overall situation.The wherein verification of data and the validity for verification data block Whether (being tampered), data block overall situation storage page number are used to indicate the data block overall situation storage page number of the data block, can be with index Data block overall situation storage page number in content is compared, to ensure the validity of data.Data actual length is for accurate Data block is obtained, is all the elements disposably read at least one memory page when due to carrying out digital independent, and data Block tends not to use entire memory page, therefore there are space junks in the memory page of part.According to Data Block Header Data actual length, the space junk in memory page can be removed from data block, so as to accurately obtain data block Content.
The process of digital independent and data write-in is carried out following using the data store organisation 20 of this preferred embodiment It will be described in the preferred embodiment of method for reading data and method for writing data, do not illustrate herein.
The only corresponding data block of each memory page of the data store organisation of this preferred embodiment, therefore only to data Can just erasing processing be carried out to the physical page, improve reading and the write operation efficiency of data block when block is deleted; The data store organisation of this preferred embodiment obtains the position of corresponding data block by data directory simultaneously, further increases number Reading according to block and write operation efficiency.
Fig. 3 is please referred to, Fig. 3 is the structural schematic diagram of the preferred embodiment of the digital independent device of the present invention.This is preferred real The digital independent device 30 for applying example may be provided in above-mentioned electronic equipment 112 comprising address acquisition module 31, index obtain Module 34 and callback module 35 are submitted in module 32, position acquisition module 33, request.Address acquisition module 31 is for obtaining data The address number of block;Acquisition module 32 is indexed to be used to, according to address number and data directory 36, obtain in the index in data directory 36 Hold, wherein index content includes that data block storage number of pages, data overall situation page number and data have mark;Position acquisition module 33 For according to data overall situation page number and hard disk quantity 37, obtaining the hard disk reel number and data block local storage page of data block Number;Request submits module 34 to be used to store number of pages, hard disk reel number and data block local storage page number according to data block, submits number According to the asynchronous read requests of block;After callback module 35 is used for operating system read block, according to asynchronous read requests, readjustment arrives In corresponding process flow.
Wherein hard disk carries out paging processing with the physical page of hard disk, to form the physical page for including at least one hard disk Hard disk memory page;Each memory page corresponds to a data block.I.e. each hard disk includes multiple memory pages, each to store The page includes at least one physical page, and physical page same in this way can only store same data block.According to the storage of data block The page generates the data block overall situation and stores page number, and generating data block according to the size of data block stores number of pages.
The digital independent device 30 of this preferred embodiment is in use, address acquisition module 31 obtains the address of data block first Number.Then the address number and data directory 36 for the data block that index acquisition module 32 is obtained according to address acquisition module 31, are obtained Take the index content in data directory 36.For the ease of quickly obtaining the data directory 36 of data block, the address number of data block It is stored in the form of key-value with corresponding index content.Rope can also be passed through by indexing the verification unit of acquisition module 32 simultaneously The data for drawing content judge whether hard disc data corresponding with index content in the presence of mark, as the data of index content exist It is identified as no, the then wrong prompt in the direct return address number of verification unit;It is identified as the data of index content exist, then school Verification certificate member returns to the data block storage number of pages of index content and the data block overall situation stores page number to position acquisition module 33.
The data block overall situation storage page number and electronics that subsequent position acquisition module 33 is obtained according to index acquisition module 32 The hard disk quantity 37 of equipment obtains the hard disk reel number and data block local storage page number of data block.Since final data are read Take is to be read from the hard disk for be stored with data block, therefore store page number and electronic equipment according to the data block overall situation of data block Hard disk quantity 37, obtain data block it is specific where hard disk and specific position (i.e. data block local storage page number).This In be written in turn with hard disk for (i.e. 1 dialer is written in 1 number block, and 2 dialers are written in 2 number blocks), it is hard such as electronic equipment Disk quantity is 8, and the data block overall situation storage page number of data block is 81, then the hard disk reel number of data block=data block overall situation storage Page number % hard disks quantity=81%8=1;Data block local storage page number=[the data block overall situation stores page number/hard disk quantity]= 8.Position acquisition module 33 can quickly obtain the hard disk reel number and data block local storage page number (number of data block in this way There is uniqueness in all data block local storage page numbers for some hard disk that electronics is set according to block local storage page number).
Then request is submitted module 34 to store number of pages, hard disk reel number and data block according to the data block of data block and is locally deposited Page number is stored up, the asynchronous read requests of data block are submitted.Wherein data block storage number of pages indicates the size of data block, the size of data block =data block stores number of pages * memory page sizes;Hard disk reel number and data block local storage page number indicate the specific of data block Position.
Finally after operating system read block, due to having got the data block, callback module 35 can be according to asking The asynchronous read requests for submitting module to submit are sought, adjusts back into corresponding process flow, continues the application process in flow. After application process i.e. in process flow sends asynchronous read requests to operating system, it can enter in the sleep queue waited for;Behaviour After system reads the data block, application process meeting Timed Recovery or readjustment are worked on using the data block.
The data read process of the digital independent device 30 of this preferred embodiment is completed in this way.
Preferably, in the digital independent device 30 of this preferred embodiment, data directory 36 may also include index stem, rope The portion of raising one's head includes the use information of hard disk, the use information of index and shared label information.The use information of hard disk includes hard Disk quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The use information of index includes having made Index quantity and not used index quantity;Shared label information is including magic number and uses label.Wherein hard disk number Amount indicates that the hard disk quantity of electronic equipment, hard disc physical number of pages indicate the quantity of the physical page of the hard disk of electronic equipment, hard disk Physical page size indicates that the size of the physical page of the hard disk of electronic equipment, current additional physical page number indicate the hard of electronic equipment The page number of the not used physical page of disk.Magic number is used to indicate validity (such as data rope of the data in the data directory portion The data drawn in portion are tampered, then the evil spirit number can change), using label for indicating that data directory use state (has made With state or unused state).The service condition of hard disk and index can be accurately obtained by the index stem user, and Data validity in data directory portion and use state.
Preferably, the stem of the data block read in the digital independent device 30 of this preferred embodiment may also include data Verification and data block overall situation storage page number and data actual length.The wherein verification of data and having for verification data block Effect property (whether being tampered), data block overall situation storage page number are used to indicate the data block overall situation storage page number of the data block, can be with Data block overall situation storage page number in index content is compared, to ensure the validity of data.Data actual length is for standard True acquisition data block is all the elements disposably read at least one memory page when due to carrying out digital independent, and Data block tends not to use entire memory page, therefore there are space junks in the memory page of part.According to data block The data actual length of stem can remove the space junk in memory page, from data block so as to accurately obtain number According to the content of block.
The digital independent device of the present invention can obtain the position of corresponding data block, and each memory page by data directory The only corresponding data block in face can accelerate the reading and write operation of data block, solve existing digital independent device and be easy resistance The technical issues of filling in application process, causing application process that can not effectively utilize the computing resource and storage resource of equipment.
Fig. 3 and Fig. 4 are please referred to, Fig. 4 is the flow chart of the preferred embodiment of the method for reading data of the present invention.This is preferred real Above-mentioned digital independent device 30 can be used to implement for the method for reading data for applying example.The method for reading data packet of this preferred embodiment It includes:
Step S401 obtains the address number of data block;
Step S402 obtains the index content in data directory according to address number and data directory;
Step S403 stores page number and hard disk quantity according to the data block overall situation, obtains the hard disk reel number of data block And data block local storage page number;
Step S404 stores number of pages, hard disk reel number and data block local storage page number according to data block, submits data block Asynchronous read requests;
Step S405, after operating system read block, according to asynchronous read requests, readjustment arrives corresponding process flow In;
The method for reading data of this preferred embodiment ends at step S405.
The following detailed description of the detailed process of each step of the method for reading data of this preferred embodiment.
In the method for reading data of this preferred embodiment, hard disk carries out paging processing with the physical page of hard disk, with shape At the memory page of the hard disk of the physical page including at least one hard disk;Each memory page corresponds to a data block.It is i.e. each Hard disk includes multiple memory pages, and each memory page includes at least one physical page, and physical page same in this way can only be deposited Store up same data block.The data block overall situation is generated according to the memory page of data block and stores page number, is generated according to the size of data block Data block stores number of pages.
In step S401, address acquisition module 31 obtains the address number of data block.Then pass to step S402.
In step S402, the address number of data block that index acquisition module 32 is obtained according to address acquisition module 31 and Data directory 36 obtains the index content in data directory 36.For the ease of quickly obtaining the data directory 36 of data block, number It is stored in the form of key-value with corresponding index content according to the address number of block.The verification list of acquisition module 32 is indexed simultaneously Member can also judge whether hard disc data corresponding with index content by the data of index content in the presence of mark, in index The data of appearance, which exist, is identified as no, the then wrong prompt in the direct return address number of verification unit;As the data of index content exist It is identified as, then verification unit returns to the data block storage number of pages of index content and data block overall situation storage page number to position is obtained Modulus block 33.Then pass to step S403.
In step S403, position acquisition module 33 stores page number according to the data block overall situation that index acquisition module 32 obtains And the hard disk quantity 37 of electronic equipment, obtain the hard disk reel number and data block local storage page number of data block.Due to final Digital independent be read from the hard disk for be stored with data block, therefore according to the data block overall situation of data block store page number and The hard disk quantity 37 of electronic equipment obtains the specific place hard disk and specific position (i.e. data block local storage of data block Page number).Here for (i.e. 1 dialer is written in 1 number block, and 2 dialers are written in 2 number blocks) being written in turn with hard disk, such as electronics The hard disk quantity of equipment is 8, and the data block overall situation storage page number of data block is 81, then hard disk reel number=data block of data block Overall situation storage page number % hard disks quantity=81%8=1;Data block local storage page number=[the data block overall situation stores page number/hard disk Quantity]=8.Position acquisition module 33 can quickly obtain the hard disk reel number and data block local storage page of data block in this way Number (the data block local storage page number has unique in all data block local storage page numbers for some hard disk that electronics is set Property).Then pass to step S404.
In step s 404, request submits module 34 to store number of pages, hard disk reel number and number according to the data block of data block According to block local storage page number, the asynchronous read requests of data block are submitted.Wherein data block storage number of pages indicates the size of data block, number Number of pages * memory page sizes are stored according to size=data block of block;Hard disk reel number and data block local storage page number indicate number According to the specific location of block.Then pass to step S405.
In step S405, after operating system read block, due to having got the data block, callback module 35 The asynchronous read requests that module is submitted can be submitted according to request, adjust back into corresponding process flow, continue in flow Application process.After application process i.e. in process flow sends asynchronous read requests to operating system, the suspend mode waited for can be entered In queue;After operating system reads the data block, application process meeting Timed Recovery or readjustment are continued using the data block Operation.
The data read process of the method for reading data of this preferred embodiment is completed in this way.
Preferably, in the method for reading data of this preferred embodiment, data directory 36 may also include index stem, index Stem includes the use information of hard disk, the use information of index and shared label information.The use information of hard disk includes hard disk Quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The use information of index includes having used Index quantity and not used index quantity;Shared label information is including magic number and uses label.Wherein hard disk quantity Indicate that the hard disk quantity of electronic equipment, hard disc physical number of pages indicate the quantity of the physical page of the hard disk of electronic equipment, hard disk object The size of the physical page of the hard disk of page size expression electronic equipment is managed, current additional physical page number indicates the hard disk of electronic equipment Not used physical page page number.Magic number is used to indicate validity (such as data directory of the data in the data directory portion Data in portion are tampered, then the evil spirit number can change), using label for indicating that data directory use state (has used State or unused state).The service condition of hard disk and index, and number can be accurately obtained by the index stem user According to the data validity and use state in index portion.
Preferably, the stem of the data block read in the method for reading data of this preferred embodiment may also include the school of data It tests and the data block overall situation stores page number and data actual length.The wherein verification of data and for the effective of verification data block Property (whether being tampered), the data block overall situation storage page number be used for indicate the data block the data block overall situation storage page number, can be with rope Draw the storage page number of the data block overall situation in content to be compared, to ensure the validity of data.Data actual length is for accurate Acquisition data block be all the elements disposably read at least one memory page, and count when due to carrying out digital independent It tends not to use entire memory page according to block, therefore there are space junks in the memory page of part.According to data block head The data actual length in portion can remove the space junk in memory page, from data block so as to accurately obtain data The content of block.
The method for reading data of the present invention can obtain the position of corresponding data block, and each memory page by data directory The only corresponding data block in face can accelerate the reading and write operation of data block, solve existing method for reading data and be easy resistance The technical issues of filling in application process, causing application process that can not effectively utilize the computing resource and storage resource of equipment.
Fig. 5 is please referred to, Fig. 5 is the structural schematic diagram of the preferred embodiment of the data transfer apparatus of the present invention.This is preferred real The data transfer apparatus 50 for applying example may be provided in above-mentioned electronic equipment 112 comprising position acquisition module 51, index generate Module 52, position generation module 53, writing module 54 and update module 55.Position acquisition module 51 is used for according to data directory 56, obtain the data block overall situation storage page number of the memory page of the blank of hard disk;Generation module 52 is indexed to be used for according to data block 57 and the data block overall situation of memory page of blank store page number, generate corresponding with data block index content and address number, Wherein index content includes that data block storage number of pages, data overall situation page number and data have mark;Position generation module 53 is used In storing page number and hard disk quantity 58 according to the data block overall situation, the hard disk reel number and data block local storage of data block are generated Page number;Writing module 54 is used to store number of pages, hard disk reel number and data block local storage page number according to data block, submits data Block write request;Update module 55 is used for according to write request as a result, updating the data index using index content.
Wherein hard disk carries out paging processing with the physical page of hard disk, to form the physical page for including at least one hard disk Hard disk memory page;Each memory page corresponds to a data block.I.e. each hard disk includes multiple memory pages, each to store The page includes at least one physical page, and physical page same in this way can only store same data block 57.According to data block 57 Memory page generates the data block overall situation and stores page number, and generating data block according to the size of data block 57 stores number of pages.
The data transfer apparatus 50 of this preferred embodiment in use, position acquisition module 51 first according to data directory 56, The data block overall situation for obtaining the memory page of the blank of hard disk stores page number.Then index generation module 52 according to data block 57 with And the data block overall situation of the memory page of blank stores page number, generates index content corresponding with data block 57 and address number.For The data directory 56 of data block 57, the address number of data block and corresponding index content are quickly obtained when read block 57 It is stored in the form of key-value.The index content includes data block storage number of pages and data overall situation page number, wherein data block The size that number of pages indicates data block 57 is stored, size=data block of data block stores number of pages * memory page sizes;Data are global Page number indicates the storage location of data block 57.
The data block overall situation storage page number and electronics that subsequent position generation module 53 is generated according to index generation module 52 The hard disk quantity 58 of equipment generates the hard disk reel number and data block local storage page number of data block.Since final data are write Fashionable is to be executed in some hard disk, therefore the hard disk quantity of page number and electronic equipment is stored according to the data block overall situation of generation 58, generate the specific write-in hard disk and specific writing position (i.e. data block local storage page number) of data block 57.Here with Hard disk is written in turn for (i.e. 1 dialer is written in 1 number block, and 2 dialers are written in 2 number blocks), such as the hard disk number of electronic equipment Amount is 8, and the data block overall situation storage page number of data block 57 is 81, then the hard disk reel number of data block=data block overall situation memory page Number % hard disks quantity=81%8=1;Data block local storage page number=[the data block overall situation stores page number/hard disk quantity]=8. Position generation module can quickly obtain the hard disk reel number of writing data blocks 57 in this way and data block local storage page number (should Data block local storage page number has uniqueness in all data block local storage page numbers for some hard disk that electronics is set).
Then writing module 54 is used to store number of pages, hard disk reel number and data block local storage page number according to data block, Submit data block write request.Wherein data block storage number of pages indicates the size of writing data blocks 57, size=number of data block Number of pages * memory page sizes are stored according to block;Hard disk reel number and data block local storage page number indicate the specific write-in of data block Position.
Final updating module 55 is according to write request as a result, updating the data index 56 using index content.Mould is such as written The data block 57 of block 54 is written successfully, then update module 55 can update the data index 56 using index content, that is, update the data rope Draw the data block storage number of pages and data block overall situation storage page number of index content corresponding with writing data blocks 57 in 56.Simultaneously There is mark in the data that the identification generation unit of update module 55 can generate index content, to indicate hard disk corresponding with index content Data are successfully written into hard disk.If failure is written in the data block of writing module 54, then update module 55 does not carry out data directory Update.
The data writing process of the data transfer apparatus 50 of this preferred embodiment is completed in this way.
Preferably, in the data transfer apparatus of this preferred embodiment, data directory 56 may also include index stem, index Stem includes the use information of hard disk, the use information of index and shared label information.The use information of hard disk includes hard disk Quantity 58, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The use information of index includes having made Index quantity and not used index quantity;Shared label information is including magic number and uses label.Wherein hard disk number Amount 58 indicates that the hard disk quantity of electronic equipment, hard disc physical number of pages indicate the quantity of the physical page of the hard disk of electronic equipment, firmly Disk physical page size indicates that the size of the physical page of the hard disk of electronic equipment, current additional physical page number indicate electronic equipment The page number of the not used physical page of hard disk.Magic number is used to indicate validity (such as data of the data in the data directory portion Data in index portion are tampered, then the evil spirit number can change), using label for having indicated data directory use state ( Use state or unused state).The service condition that hard disk and index can be accurately obtained by the index stem user, with And the data validity in data directory portion and use state.
Preferably, it may also include data in the stem of the data block 57 of the data transfer apparatus of this preferred embodiment write-in Verification and data block overall situation storage page number and data actual length.The wherein verification of data and for verification data block 57 Whether validity (is tampered), and data block overall situation storage page number is used to indicate the data block overall situation storage page number of the data block, can It is compared with the data block overall situation storage page number in index content, to ensure the validity of data.Data actual length is used for Data block 57 is accurately obtained, when due to carrying out digital independent, is disposably read in all at least one memory page Hold, and data block 57 tends not to use entire memory page, therefore there are space junks in the memory page of part.According to The data actual length of Data Block Header can remove the space junk in memory page from data block, to read the number According to when can accurately obtain the content of data block 57.
The data transfer apparatus of the present invention can generate the position of corresponding data block by establishing data directory, and each deposit The only corresponding data block of the storage page can accelerate the reading and write operation of data block, solve existing method for reading data and hold Easily obstruction application process, causes application process not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
Fig. 5 and Fig. 6 are please referred to, Fig. 6 is the structural schematic diagram of the preferred embodiment of the data transfer apparatus of the present invention.This is excellent Select the method for writing data of embodiment that above-mentioned data transfer apparatus 50 can be used to implement.The data write-in side of this preferred embodiment Method includes:
Step S601 obtains the data block overall situation storage page number of the memory page of the blank of hard disk according to data directory;
Step S602 stores page number, generation and data according to the data block overall situation of data block and the memory page of blank The corresponding index content of block and address number;
Step S603 stores page number and hard disk quantity according to the data block overall situation, generate data block hard disk reel number and Data block local storage page number;
Step S604 stores number of pages, hard disk reel number and data block local storage page number according to data block, submits data block Write request;
Step S605, according to write request as a result, updating the data index using index content;
The method for writing data of this preferred embodiment ends at step S605.
The following detailed description of the detailed process of each step of the method for writing data of this preferred embodiment.
In the method for writing data of this preferred embodiment, hard disk carries out paging processing with the physical page of hard disk, with shape At the memory page of the hard disk of the physical page including at least one hard disk;Each memory page corresponds to a data block.It is i.e. each Hard disk includes multiple memory pages, and each memory page includes at least one physical page, and physical page same in this way can only be deposited Store up same data block.The data block overall situation is generated according to the memory page of data block and stores page number, is generated according to the size of data block Data block stores number of pages.
In step s 601, position acquisition module 51 obtains the memory page of the blank of hard disk according to data directory 56 The data block overall situation stores page number.Then pass to step S602.
In step S602, index generation module 52 is according to the data block of data block 57 and the memory page of the blank overall situation Page number is stored, index content corresponding with data block 57 and address number are generated.In order to quickly obtain number when read block 57 According to the data directory 56 of block 57, the address number of data block 57 is stored with corresponding index content in the form of key-value.The rope It includes data block storage number of pages and data overall situation page number to draw content, and wherein data block storage number of pages indicates the big of data block 57 Small, size=data block of data block stores number of pages * memory page sizes;Data overall situation page number indicates the storage position of data block 57 It sets.Then pass to step S603.
In step S603, position generation module 53 stores page number according to the data block overall situation that index generation module 52 generates And the hard disk quantity 58 of electronic equipment, generate the hard disk reel number and data block local storage page number of data block.Due to final Data write-in when be to be executed in some hard disk, therefore page number and electronic equipment are stored according to the data block overall situation of generation Hard disk quantity 58 generates the specific write-in hard disk and specific writing position (i.e. data block local storage page of data block 57 Number).Here for (i.e. 1 dialer is written in 1 number block, and 2 dialers are written in 2 number blocks) being written in turn with hard disk, as electronics is set Standby hard disk quantity is 8, and the data block overall situation storage page number of data block is 81, then hard disk reel number=data block of data block is complete Office's storage page number % hard disks quantity=81%8=1;Data block local storage page number=[the data block overall situation stores page number/hard disk number Amount]=8.Position generation module can quickly obtain the hard disk reel number and data block local storage of writing data blocks 57 in this way (the data block local storage page number has only page number in all data block local storage page numbers for some hard disk that electronics is set One property).Then pass to step S604.
In step s 604, writing module 54 is used to store number of pages, hard disk reel number and data block part according to data block Page number is stored, data block write request is submitted.Wherein data block storage number of pages indicates the size of writing data blocks 57, data block Size=data block stores number of pages * memory page sizes;Hard disk reel number and data block local storage page number indicate data block Specific writing position.Then pass to step S605.
In step s 605, update module 55 according to write request as a result, updating the data index 56 using index content. If the data block 57 of writing module 54 is written successfully, then update module 55 can update the data index 56 using index content, i.e., more The data block storage number of pages of index content corresponding with writing data blocks 57 and the storage of the data block overall situation in new data index 56 Page number.There is mark in the data that the identification generation unit of update module 55 can generate index content simultaneously, in indicating and index Hold corresponding hard disc data and is successfully written into hard disk.If failure is written in the data block of writing module 54, then the not logarithm of update module 55 It is updated according to index.
The data writing process of the method for writing data of this preferred embodiment is completed in this way.
Preferably, in the method for writing data of this preferred embodiment, data directory 56 may also include index stem, index Stem includes the use information of hard disk, the use information of index and shared label information.The use information of hard disk includes hard disk Quantity 58, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The use information of index includes having made Index quantity and not used index quantity;Shared label information is including magic number and uses label.Wherein hard disk number Amount 58 indicates that the hard disk quantity of electronic equipment, hard disc physical number of pages indicate the quantity of the physical page of the hard disk of electronic equipment, firmly Disk physical page size indicates that the size of the physical page of the hard disk of electronic equipment, current additional physical page number indicate electronic equipment The page number of the not used physical page of hard disk.Magic number is used to indicate validity (such as data of the data in the data directory portion Data in index portion are tampered, then the evil spirit number can change), using label for having indicated data directory use state ( Use state or unused state).The service condition that hard disk and index can be accurately obtained by the index stem user, with And the data validity in data directory portion and use state.
Preferably, it may also include data in the stem of the data block 57 of the method for writing data of this preferred embodiment write-in Verification and data block overall situation storage page number and data actual length.The wherein verification of data and for verification data block 57 Whether validity (is tampered), and data block overall situation storage page number is used to indicate the data block overall situation storage page number of the data block, can It is compared with the data block overall situation storage page number in index content, to ensure the validity of data.Data actual length is used for Data block 57 is accurately obtained, when due to carrying out digital independent, is disposably read in all at least one memory page Hold, and data block 57 tends not to use entire memory page, therefore there are space junks in the memory page of part.According to The data actual length of Data Block Header can remove the space junk in memory page from data block, to read the number According to when can accurately obtain the content of data block 57.
The method for writing data of the present invention can generate the position of corresponding data block by establishing data directory, and each deposit The only corresponding data block of the storage page can accelerate the reading and write operation of data block, solve existing method for reading data and hold Easily obstruction application process, causes application process not asked using the technology of the computing resource of equipment and storage resource effectively Topic.
Fig. 7 is please referred to, Fig. 7 is the flow chart of the method for writing data of the present invention and the specific embodiment of device.Lead to below It crosses specific embodiment shown in Fig. 7 and illustrates the method for writing data of this preferred embodiment and the concrete operating principle of device.
Step S701, application process stores page number according to the data block overall situation of data block and the memory page of blank, raw At index content corresponding with data block and address number, the range of address number can be 0 to (index total quantity -1).
Step S702, according to index content and the hard disk quantity of electronic equipment, generate data block hard disk reel number and Data block local storage page number, that is, determine the size and location of data block.
Step S703, handles data block, i.e., verification and the data block overall situation of data are arranged in the stem of data block Store page number and data actual length.
Step S704, will treated data block, by the hard disk reel number and data block local storage of the data block generated Page number is sequentially written in hard disk.
Step S705, according to write-in as a result, a pair index content corresponding with address number is updated.
Fig. 8 is please referred to, Fig. 8 is the flow chart of the method for reading data of the present invention and the specific embodiment of device.Lead to below It crosses specific embodiment shown in Fig. 8 and illustrates the method for reading data of this preferred embodiment and the concrete operating principle of device.
Step S801, according to the address number of data block, the data block obtained in index content stores number of pages, data overall situation page Number and data exist mark.
Step S802 exists according to data and identifies, judges whether the corresponding hard disc data of index content is written.
Step S803 then directly returns to error code if data are no in the presence of being identified as;It is identified as, then will as data exist Data block overall situation storage page number is converted to the hard disk reel number and data block local storage page number of data block.
Step S804 stores number of pages, hard disk reel number and data block local storage page number according to the data block of data block, to Operating system submits asynchronous read requests.
Step S805, corresponding process flow enter in the sleep queue of waiting;
Step S806, after operating system has handled the asynchronous read requests, in readjustment to corresponding process flow.
Table 2 is the concurrency situation of the server after digital independent, wiring method and device using the present invention.
Table 2
Server A concurrency Server B concurrency
Synchronous read-write 350 300
Asynchronous read and write 610 540
The present invention 700 620
As seen from the above table, digital independent of the invention, wiring method and device and data store organisation are adequately utilized The computing resource and storage resource of hardware server, when solving existing read operation and write operation easily obstruction application into The problem of journey, improve the performance of server, effect better than simple synchronous read-write and asynchronous read and write, realize server into The high concurrent and height of journey are handled up.
Digital independent, wiring method and the device and data store organisation of the present invention can be obtained corresponding by data directory The position of data block, and the only corresponding data block of each memory page can accelerate the reading and write operation of data block, solve Existing digital independent, wiring method and device are easy obstruction application process, cause the application process not utilize effectively The technical issues of computing resource and storage resource of equipment.
Each functional unit in the embodiment of the present invention can be integrated in a processing module, can also be each unit list Solely be physically present, can also two or more units be integrated in a module.Above-mentioned integrated module both may be used The form of hardware is realized, can also be realized in the form of software function module.If the integrated module is with software function The form of module is realized and when sold or used as an independent product, can also be stored in a computer-readable storage and be situated between In matter.Storage medium mentioned above can be read-only memory, disk or CD etc..Above-mentioned each device or system, can be with Execute the method in correlation method embodiment.
In conclusion although the present invention is disclosed above with preferred embodiment, above preferred embodiment is not to limit The system present invention, those skilled in the art can make various changes and profit without departing from the spirit and scope of the present invention Decorations, therefore protection scope of the present invention is subject to the range that claim defines.

Claims (25)

1. a kind of method for reading data, which is characterized in that including step:
Obtain the address number of data block;
According to described address number and data directory, the index content in the data directory is obtained, wherein the index content packet Include data block storage number of pages and data block overall situation storage page number;
Page number and hard disk quantity are stored according to the data block overall situation, obtains the hard disk reel number and data block of the data block Local storage page number;Wherein the hard disk reel number of data block is that the data block overall situation stores page number % hard disk quantity;Data block is locally deposited It is that the data block overall situation stores page number/hard disk quantity to store up page number;
Number of pages, the hard disk reel number and the data block local storage page number are stored according to the data block, submits data block Asynchronous read requests;And
After operating system read block, according to the asynchronous read requests, adjust back into corresponding process flow;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one hard disk to be formed The memory page of the hard disk of physical page;Each memory page corresponds to a data block;According to the data block Memory page generate the data block overall situation and store page number, the data block memory page is generated according to the size of the data block Number;The size of data block=data block stores number of pages * memory page sizes.
2. method for reading data according to claim 1, which is characterized in that the address number of the data block and corresponding rope Draw content to be stored in the form of key-value.
3. method for reading data according to claim 1, which is characterized in that the index content further includes being for judgement There is mark in the no data that there is hard disc data corresponding with the index content;
The step of index content obtained in the data directory includes:
The data such as the index content are no in the presence of being identified as, then return to described address mistake;And such as the index The data of content, which exist, to be identified as, then returns to the data block storage number of pages of the index content and the data The block overall situation stores page number.
4. method for reading data according to claim 1, which is characterized in that the stem of the data block includes the school of data It tests and the data block overall situation stores page number and data actual length.
5. method for reading data according to claim 1, which is characterized in that the data directory further includes index stem, The index stem includes the use information of hard disk, the use information of index and shared label information;The use of the hard disk Information includes hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently adds physical page number;The index makes Include used index quantity and not used index quantity with information;The shared label information includes having for verifying Effect property magic number and for indicate the data directory use state use mark.
6. a kind of method for writing data, which is characterized in that including step:
According to data directory, the data block overall situation storage page number of the memory page of the blank of hard disk is obtained;
Page number is stored according to the data block overall situation of the data block and the memory page of the blank, is generated and the data block Corresponding index content and address number, wherein the index content includes data block storage number of pages and data block overall situation memory page Number;
Page number and hard disk quantity are stored according to the data block overall situation, generates the hard disk reel number and data block of the data block Local storage page number;Wherein the hard disk reel number of data block is that the data block overall situation stores page number % hard disk quantity;Data block is locally deposited It is that the data block overall situation stores page number/hard disk quantity to store up page number;
Number of pages, the hard disk reel number and the data block local storage page number are stored according to the data block, submits data block Write request;And
According to said write request as a result, updating the data directory using the index content;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one hard disk to be formed The memory page of the hard disk of physical page;Each memory page corresponds to a data block;According to the data block Memory page generate the data block overall situation and store page number, the data block memory page is generated according to the size of the data block Number;The size of data block=data block stores number of pages * memory page sizes.
7. method for writing data according to claim 6, which is characterized in that the address number of the data block and corresponding rope Draw content to be stored in the form of key-value.
8. method for writing data according to claim 6, which is characterized in that the index content further includes being for judgement There is mark in the no data that there is hard disc data corresponding with the index content;
The method for writing data further includes step:
According to said write request as a result, the data for generating the index content have mark.
9. method for writing data according to claim 6, which is characterized in that the method for writing data further includes step:
The data block is handled, to generate the stem of the data block;The stem of the wherein described data block includes data Verification and data block overall situation storage page number and data actual length.
10. method for writing data according to claim 6, which is characterized in that the data directory further includes index stem, The index stem includes the use information of hard disk, the use information of index and shared label information;The use of the hard disk Information includes hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently adds physical page number;The index makes Include used index quantity and not used index quantity with information;The shared label information includes having for verifying Effect property magic number and for indicate the data directory use state use mark.
11. a kind of data store organisation, which is characterized in that including:
Data store, including multiple hard disks, the hard disk carry out paging processing with the physical page of the hard disk, to form packet Include the memory page of the hard disk of the physical page of at least one hard disk;Wherein each memory page corresponds to a number According to block;
Data directory portion, including index content, including:
Data block stores number of pages, the size for indicating the data block;And
The data block overall situation stores page number, the storage location for indicating the data block;And
Address number, it is corresponding with the index content, for being supplied to user, with read block.
12. data store organisation according to claim 11, which is characterized in that the address number of the data block and corresponding Index content is stored in the form of key-value.
13. data store organisation according to claim 11, which is characterized in that the index content further includes:
There is mark in data, for judging whether hard disc data corresponding with the index content.
14. data store organisation according to claim 11, which is characterized in that the stem of the data block includes data Verification and data block overall situation storage page number and data actual length.
15. data store organisation according to claim 11, which is characterized in that the data directory portion further includes that index is first Portion, the index stem includes the use information of hard disk, the use information of index and shared label information;The hard disk makes With information include hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The index Use information includes used index quantity and not used index quantity;The shared label information includes for verifying The magic number of validity and for indicate the data directory use state use mark.
16. a kind of digital independent device, which is characterized in that including:
Address acquisition module, the address number for obtaining data block;
Acquisition module is indexed, for according to described address number and data directory, obtaining the index content in the data directory, Described in index content include data block storage number of pages and the data block overall situation storage page number;
Position acquisition module obtains the data block for storing page number and hard disk quantity according to the data block overall situation Hard disk reel number and data block local storage page number;Wherein the hard disk reel number of data block is that the data block overall situation stores page number % hard disks Quantity;Data block local storage page number is that the data block overall situation stores page number/hard disk quantity;
Module is submitted in request, is locally deposited for storing number of pages, the hard disk reel number and the data block according to the data block Page number is stored up, the asynchronous read requests of data block are submitted;And
Callback module, for after operating system read block, according to the asynchronous read requests, readjustment to arrive corresponding processing stream Cheng Zhong;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one hard disk to be formed The memory page of the hard disk of physical page;Each memory page corresponds to a data block;According to the data block Memory page generate the data block overall situation and store page number, the data block memory page is generated according to the size of the data block Number;The size of data block=data block stores number of pages * memory page sizes.
17. digital independent device according to claim 16, which is characterized in that the address number of the data block and corresponding Index content is stored in the form of key-value.
18. digital independent device according to claim 16, which is characterized in that the index content further includes for judging There is mark with the presence or absence of the data of hard disc data corresponding with the index content;
The index acquisition module includes:
Verification unit, the data for such as described index content are no in the presence of being identified as, then return to described address mistake;Such as The data of the index content, which exist, to be identified as, then return the index content data block storage number of pages and The data block overall situation stores page number.
19. digital independent device according to claim 16, which is characterized in that the stem of the data block includes data Verification and data block overall situation storage page number and data actual length.
20. digital independent device according to claim 16, which is characterized in that the data directory further includes that index is first Portion, the index stem includes the use information of hard disk, the use information of index and shared label information;The hard disk makes With information include hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The index Use information includes used index quantity and not used index quantity;The shared label information includes for verifying The magic number of validity and for indicate the data directory use state use mark.
21. a kind of data transfer apparatus, which is characterized in that including:
Position acquisition module, for according to data directory, obtaining the data block overall situation memory page of the memory page of the blank of hard disk Number;
Generation module is indexed, for the data block overall situation memory page according to the data block and the memory page of the blank Number, index content corresponding with the data block and address number are generated, wherein the index content includes data block storage number of pages And the data block overall situation stores page number;
Position generation module generates the data block for storing page number and hard disk quantity according to the data block overall situation Hard disk reel number and data block local storage page number;Wherein the hard disk reel number of data block is that the data block overall situation stores page number % hard disks Quantity;Data block local storage page number is that the data block overall situation stores page number/hard disk quantity;
Writing module, for storing number of pages, the hard disk reel number and the data block local storage page according to the data block Number, submit data block write request;And
Update module, for according to said write ask as a result, using the index content update data directory;
The wherein described hard disk carries out paging processing with the physical page of the hard disk, includes at least one hard disk to be formed The memory page of the hard disk of physical page;Each memory page corresponds to a data block;According to the data block Memory page generate the data block overall situation and store page number, the data block memory page is generated according to the size of the data block Number;The size of data block=data block stores number of pages * memory page sizes.
22. data transfer apparatus according to claim 21, which is characterized in that the address number of the data block and corresponding Index content is stored in the form of key-value.
23. data transfer apparatus according to claim 21, which is characterized in that the index content further includes for judging There is mark with the presence or absence of the data of hard disc data corresponding with the index content;
The update module includes:
Identification generation unit, for according to said write ask as a result, generate the index content the data exist mark Know.
24. data transfer apparatus according to claim 21, which is characterized in that the stem of the data block includes data Verification and data block overall situation storage page number and data actual length.
25. data transfer apparatus according to claim 21, which is characterized in that the data directory further includes that index is first Portion, the index stem includes the use information of hard disk, the use information of index and shared label information;The hard disk makes With information include hard disk quantity, hard disc physical number of pages, hard disc physical page size and currently add physical page number;The index Use information includes used index quantity and not used index quantity;The shared label information includes for verifying The magic number of validity and for indicate the data directory use state use mark.
CN201310534962.0A 2013-10-31 2013-10-31 Digital independent, wiring method and device and data store organisation Active CN104598161B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310534962.0A CN104598161B (en) 2013-10-31 2013-10-31 Digital independent, wiring method and device and data store organisation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310534962.0A CN104598161B (en) 2013-10-31 2013-10-31 Digital independent, wiring method and device and data store organisation

Publications (2)

Publication Number Publication Date
CN104598161A CN104598161A (en) 2015-05-06
CN104598161B true CN104598161B (en) 2018-10-30

Family

ID=53123989

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310534962.0A Active CN104598161B (en) 2013-10-31 2013-10-31 Digital independent, wiring method and device and data store organisation

Country Status (1)

Country Link
CN (1) CN104598161B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017068617A1 (en) * 2015-10-19 2017-04-27 株式会社日立製作所 Storage system
CN108427675B (en) * 2017-02-13 2022-09-02 阿里巴巴集团控股有限公司 Method and equipment for constructing index
CN107391030B (en) * 2017-06-26 2020-04-28 日立楼宇技术(广州)有限公司 Data storage method, data storage device, computer-readable storage medium and computer equipment
CN107844273B (en) * 2017-11-01 2021-05-04 深信服科技股份有限公司 Data writing method and device and verification method and device
CN108829571B (en) * 2018-05-23 2021-07-27 四川巧夺天工信息安全智能设备有限公司 Method for extracting dormant data of Windows operating system
CN111309471B (en) * 2018-12-11 2024-02-09 迈普通信技术股份有限公司 Data processing method, device and distributed system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1786893A (en) * 2004-12-10 2006-06-14 国际商业机器公司 Transferring data among a logical layer, physical layer, and storage device
CN101256535A (en) * 2007-03-01 2008-09-03 创惟科技股份有限公司 Apparatus and method for reading-writing and rapidly fetching hard disk data
CN101620517A (en) * 2009-08-04 2010-01-06 成都市华为赛门铁克科技有限公司 Data-writing method and data-writing device
CN103207836A (en) * 2012-01-16 2013-07-17 百度在线网络技术(北京)有限公司 Write method and write device for solid-state storage hard disk
CN103257831A (en) * 2012-02-20 2013-08-21 深圳市腾讯计算机系统有限公司 Reading-writing control method of storage and corresponding storage

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110276744A1 (en) * 2010-05-05 2011-11-10 Microsoft Corporation Flash memory cache including for use with persistent key-value store

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1786893A (en) * 2004-12-10 2006-06-14 国际商业机器公司 Transferring data among a logical layer, physical layer, and storage device
CN101256535A (en) * 2007-03-01 2008-09-03 创惟科技股份有限公司 Apparatus and method for reading-writing and rapidly fetching hard disk data
CN101620517A (en) * 2009-08-04 2010-01-06 成都市华为赛门铁克科技有限公司 Data-writing method and data-writing device
CN103207836A (en) * 2012-01-16 2013-07-17 百度在线网络技术(北京)有限公司 Write method and write device for solid-state storage hard disk
CN103257831A (en) * 2012-02-20 2013-08-21 深圳市腾讯计算机系统有限公司 Reading-writing control method of storage and corresponding storage

Also Published As

Publication number Publication date
CN104598161A (en) 2015-05-06

Similar Documents

Publication Publication Date Title
CN104598161B (en) Digital independent, wiring method and device and data store organisation
CN105224237A (en) A kind of date storage method and device
CN104809179A (en) Device and method for accessing Hash table
CN104461400B (en) The method and apparatus for handling access request conflict
CN108027713A (en) Data de-duplication for solid state drive controller
CN109871182A (en) Storage device and its operating method and the method for issuing order
US20200050558A1 (en) Modifying NVMe Physical Region Page List Pointers and Data Pointers to Facilitate Routing of PCIe Memory Requests
US20120117347A1 (en) Initializing of a memory area
CN109271110A (en) The wiring method of storing data, writing system and relevant apparatus in storage system
CN109460406A (en) A kind of data processing method and device
CN114385089B (en) Cross addressing-based dynamic bank storage method and device and electronic equipment
CN109213691B (en) Method and apparatus for cache management
CN109947669A (en) The address conversion method and system of KV storage equipment
US11030714B2 (en) Wide key hash table for a graphics processing unit
CN106598548A (en) Solution method and device for read-write conflict of storage unit
US9021321B2 (en) Testing disk drives shared by multiple processors in a supercomputer complex
WO2014190700A1 (en) Method of memory access, buffer scheduler and memory module
CN111782153A (en) Method, system, device and medium based on read data
CN116842012A (en) Method, device, equipment and storage medium for storing Redis cluster in fragments
CN104572638A (en) Data reading and writing method and device
CN116457751A (en) Write data access structure and chip
US20230198770A1 (en) Consensus Data Filtering in Proof of Space Blockchain Systems
CN109582226A (en) A kind of high speed storing access logical construction and its control method
CN107003932A (en) The CACHE DIRECTORY processing method and contents controller of multi-core processor system
CN111104435B (en) Metadata organization method, device and equipment and computer readable storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant