CN104461943A - Data reading method, device and system - Google Patents
Data reading method, device and system Download PDFInfo
- Publication number
- CN104461943A CN104461943A CN201410836741.3A CN201410836741A CN104461943A CN 104461943 A CN104461943 A CN 104461943A CN 201410836741 A CN201410836741 A CN 201410836741A CN 104461943 A CN104461943 A CN 104461943A
- Authority
- CN
- China
- Prior art keywords
- data
- read
- buffer memory
- address information
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a data reading method, device and system, and relates to the field of data reading. The data reading system comprises a data request end, a cache and a storage node storing data. The data reading device is arranged in the cache, and the data are read through a data pre-cache method. Different from the prior art that a cache starts to read data from a storage node after the read data are received, a reading request instruction sent to the cache from the data request end includes number information of the data needing to be read and the address information of the data needing to be read, the cache starts to carry out pre-reading from the storage node after receiving the read request instruction, and the data request end can directly read the needed data from the cache. By means of the mode, the reading hit rate is increased, and the data reading efficiency is further improved.
Description
Technical field
The present invention relates to field of data storage, in particular to method for reading data, device and system.
Background technology
The flow process that current common data reads is, request of data end sends and reads request of data, read request of data to be received by buffer memory, and the response returned reading request of data, after request of data termination receives response, then send the instruction of reading data to buffer memory, if there are the data needing to read in buffer memory, then request of data end directly reads from buffer memory, if do not had, reads from hard disk.When the data needing to read are larger, data can be divided into many parts by request of data end, by a batch reading, after having read first data, then start to read second batch data, until read all data.Adopt such data reading mode, cache hit rate is not high, and whole read requests affairs are serial processing mechanism, and speed is slow.
Summary of the invention
The object of the present invention is to provide a kind of method for reading data, device and system, the problem that when reading data at present to improve, cache hit rate is high, not slow.
To achieve these goals, the technical scheme of embodiment of the present invention employing is as follows:
First aspect, embodiments provides a kind of method for reading data, is applied to data reading system, and described data reading system comprises request of data end, buffer memory and stores the memory node of data, and described method comprises:
Described buffer memory receives the read requests instruction that described request of data end sends, and described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Send response instruction to described request of data end, in described buffer memory, search the described data needing to read;
If there is no the described data needing to read in described buffer memory, start pre-buffering task, read data corresponding to described address information according to the described address information of the data read that needs from described memory node;
Described buffer memory starts data and returns task, and the data read from described memory node are returned described request of data end;
Described pre-buffering task and described data return task and perform simultaneously, separate.
In conjunction with first aspect, embodiments provide the first possible embodiment of first aspect, wherein, describedly read data corresponding to described address information from described memory node, comprising: once read data corresponding to whole described address information from described memory node;
Described the data read from described memory node are returned described request of data end, comprising: data corresponding for the whole described address information read from described memory node are once returned described request of data end.
In conjunction with first aspect, embodiments provide the embodiment that the second of first aspect is possible, wherein, described buffer memory divides multiple batches the data that described needs read are returned described request of data end, described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node, if described buffer memory one batch return data amount can be more than or equal to the data volume needing the address information of the data read corresponding described in a group
Describedly read data corresponding to described address information from described memory node, comprising: read from described memory node the data that described in a group, address information is corresponding;
Described the data read from described memory node are returned described request of data end, comprising:
Data corresponding for address information described in described a group are returned described request of data end by described buffer memory;
Described method also comprises: repeat said process, until the described data read that need all read and return described request of data end by described buffer memory.
In conjunction with the embodiment that the second of first aspect is possible, embodiments provide the third possible embodiment of first aspect, wherein, if the data volume that the data of batch can return is less than the data volume that described in a group, address information is corresponding,
Data corresponding for address information described in described a group are returned described request of data end by described buffer memory, comprising:
The data of described one batch are returned described request of data end by described buffer memory, repeat this step, until the data that described in a group, address information is corresponding all return described request of data end.
Second aspect, the embodiment of the present invention additionally provides, a kind of data fetch device, it is characterized in that, be applied to data reading system, described data reading system comprises request of data end, buffer memory and stores the memory node of data, and described data fetch device is arranged at described buffer memory, and described device comprises:
Read requests instruction reception unit, for receiving the read requests instruction that described request of data end sends, described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Response instruction sending unit, for sending response instruction to described request of data end, and searches the described data needing to read in described buffer memory;
Storing data-reading unit, if for there is no the described data needing to read in described buffer memory, start pre-buffering task, reading data corresponding to described address information according to the described address information of the data read that needs from described memory node;
Data return unit, return task for starting data, and the data read from described memory node are returned described request of data end.
In conjunction with second aspect, embodiments provide the first possible embodiment of second aspect, wherein, described storage data-reading unit also for: once read data corresponding to whole described address information from described memory node;
Described data return unit also for: data corresponding for the whole described address information read from described memory node are once returned described request of data end.
In conjunction with second aspect, embodiments provide the embodiment that the second of second aspect is possible, wherein, described buffer memory divides multiple batches the data that described needs read are returned described request of data end, described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node, if described buffer memory one batch return data amount can be more than or equal to the data volume needing the address information of the data read corresponding described in a group
Described storage data-reading unit also for: read from described memory node the data that described in a group, address information is corresponding;
Described data return unit also for: data corresponding for address information described in described a group are returned described request of data end by described buffer memory.
In conjunction with the embodiment that the second of second aspect is possible, embodiments provide the third possible embodiment of second aspect, wherein, if the data volume that the data of batch can return is less than the data volume that described in a group, address information is corresponding,
Described data return unit also for: the data of described batch are returned described request of data end by described buffer memory, repeat this step, until the data that described in a group, address information is corresponding all return described request of data end.
The third aspect, the embodiment of the present invention additionally provides a kind of data reading system, and described data reading system comprises request of data end, buffer memory and stores the memory node of data,
Described request of data end sends read requests instruction to described buffer memory;
Described buffer memory receives the read requests instruction that described request of data end sends, and described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Described buffer memory sends response instruction to described request of data end, searches the described data needing to read in described buffer memory; If do not have the described data needing to read in described buffer memory, according to the described address information needing the data read, described buffer memory reads data corresponding to described address information from described memory node;
The data read from described memory node are returned described request of data end by described buffer memory.
In conjunction with the third aspect, embodiments provide the first possible embodiment of the third aspect, wherein, described data reading system is applied to distributed memory system.
The method for reading data that the embodiment of the present invention provides, device and system, the method of data pre-buffering is adopted to carry out the reading of data, from be buffered in prior art receive digital independent after just to start to read data from memory node different, the address information of the data that the quantity information of the data that needs read and described needs read is included in the read requests instruction that request of data end sends to buffer memory, be buffered in after receiving read requests instruction and just start to carry out pre-read from memory node, request of data end directly just can read the data of needs from buffer memory, adopt in such a way, improve and read hit rate, further increase the reading efficiency of data.
For making above-mentioned purpose of the present invention, feature and advantage become apparent, preferred embodiment cited below particularly, and coordinate appended accompanying drawing, be described in detail below.
Accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme of the embodiment of the present invention, be briefly described to the accompanying drawing used required in embodiment below, be to be understood that, the following drawings illustrate only some embodiment of the present invention, therefore the restriction to scope should be counted as, for those of ordinary skill in the art, under the prerequisite not paying creative work, other relevant accompanying drawings can also be obtained according to these accompanying drawings.Shown in accompanying drawing, above-mentioned and other object of the present invention, Characteristics and advantages will be more clear.Reference numeral identical in whole accompanying drawing indicates identical part.Deliberately do not draw accompanying drawing by physical size equal proportion convergent-divergent, focus on purport of the present invention is shown.
Fig. 1 shows the structured flowchart of a kind of data reading system that the embodiment of the present invention provides;
Fig. 2 shows the process flow diagram of a kind of method for reading data that the embodiment of the present invention provides;
Fig. 3 shows the process flow diagram of the another kind of method for reading data that the embodiment of the present invention provides;
Fig. 4 shows the structured flowchart of a kind of data fetch device that the embodiment of the present invention provides;
Fig. 5 shows the structured flowchart of the another kind of data synchronization unit that the embodiment of the present invention provides.
Embodiment
Below in conjunction with accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.The assembly of the embodiment of the present invention describing and illustrate in usual accompanying drawing herein can be arranged with various different configuration and design.Therefore, below to the detailed description of the embodiments of the invention provided in the accompanying drawings and the claimed scope of the present invention of not intended to be limiting, but selected embodiment of the present invention is only represented.Based on embodiments of the invention, the every other embodiment that those skilled in the art obtain under the prerequisite not making creative work, all belongs to the scope of protection of the invention.
Cloud storage is in the conceptive extension of cloud computing and the new concept of development out one, refer to by functions such as cluster application, network technology or distributed file systems, various dissimilar memory device a large amount of in network is gathered collaborative work, a system of data storage and Operational Visit function is externally provided jointly.When the core of cloud computing system computing and process is the store and management of mass data, just need in cloud computing system to configure a large amount of memory devices, so cloud computing system is just transformed into a cloud storage system, so the cloud computing system that cloud storage is is core with data store and management.In simple terms, cloud stores is exactly the emerging scheme of one be put into by storage resources for people's access on cloud.User can at any time, Anywhere, to be connected on cloud access data easily through any web-enabled device.
The flow process of the common digital independent be applied in cloud storage system is at present, request of data end sends read requests instruction, read requests can be received by buffer memory, and the response returned reading request of data, after request of data termination receives response, then send the instruction of reading data to buffer memory, if there are the data needing to read in buffer memory, then request of data end directly reads from buffer memory, if do not had, reads from hard disk.When the data needing to read are larger, data can be divided into many parts by request of data end, by by a batch reading, after having read first data, then start to read second batch data, until read all data.Inventor finds through long-term observation and research, and cache hit rate is not high, and whole read requests affairs are serial processing mechanism, and speed is slow.
Invention people improves for making existing situation, provides the embodiment of the present invention.The address information of the data that the quantity information of the data that needs read and described needs read is included in the read requests instruction that request of data end sends to buffer memory, be buffered in after receiving read requests instruction and just start to carry out pre-read from memory node, when request of data end sends the instruction of reading described data, buffer memory will need all or part of pre-read of the data read, request of data end directly just can read the data of needs from buffer memory, adopt in such a way, improve and read hit rate, further increase the reading efficiency of data.
See Fig. 1, the embodiment of the present invention is applied to data reading system, and described data reading system comprises request of data end 100, buffer memory 200 and stores the memory node 300 of data.
See Fig. 2, embodiments provide a kind of method for reading data, described method comprises:
Step 101: described buffer memory receives the read requests instruction that described request of data end sends, described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading.
Be buffered in the quantity information and address information that are also aware of the data needing reading while receiving read requests instruction, wherein, described buffer memory can be the caching server in distributed storage system, and can be also other equipment, the embodiment of the present invention limit.
Step 102: described buffer memory sends response instruction to described request of data end.
Buffer memory sends response instruction to described request of data end, represents the operation that can perform and read data.
Step 103: described in be buffered in described buffer memory search described need read data.
Request of data end reads data velocity ratio from buffer memory reads from memory node wants fast, therefore preferentially search the described data needing to read in the buffer, if had, just by buffer memory directly by this data return data request end.
Step 104: if do not have the described data needing to read in described buffer memory, start pre-buffering task, reads data corresponding to described address information according to the described address information of the data read that needs from described memory node.
Buffer memory has obtained the address information needing the data read from described read requests instruction, and after receiving described read requests instruction, buffer memory reads needing the data read from described memory node with regard to starting immediately.
Step 105: described buffer memory starts data and returns task, and the data read from described memory node are once returned described request of data end.
Starting to read needing the data read from described memory node being buffered in, just immediately the data read being returned data request end simultaneously.
Described pre-buffering task and data return task and perform simultaneously, separate, buffer memory need not be waited for that digital independent is complete and just start return data, has data to read and just starts return data immediately, and cache read is fetched data and the operation of return data is divided into two task pipelines to carry out simultaneously
In the present embodiment, buffer memory reads data and buffer memory by data return data request end from described memory node, and what all adopt is that sliding window mechanism is carried out.
Such as: request of data end needs the data of reading 10 pages, and request of data end sends read requests instruction to buffer memory, this read requests instruction comprises the address information of the data of the reading 10 pages of needs.Be buffered in after receiving read requests instruction, response instruction is sent to request of data end, search spatial cache and whether have the data needing to read, if do not had, the address information of the data of the reading 10 pages of the needs comprised according to read requests instruction, once the data of 10 pages are all run through from memory node, and be stored in spatial cache.After the instruction of data receiving the reading 10 pages that request of data end sends, these data of 10 pages are once all returned described request of data end.
The method for reading data that the present embodiment provides, utilize buffer memory buffer memory return data to the time of request of data end, simultaneously to needing the data read to carry out pre-read, pre-buffering task and data return task and perform simultaneously, separate, increase read request hit rate, improve reading rate, save the time of reading from memory node.
See Fig. 3, embodiments provide another kind of method for reading data, be applied to data reading system, described data reading system comprises request of data end, buffer memory and stores the memory node of data, and described method comprises.
Step 201: described buffer memory receives the read requests instruction that described request of data end sends, described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading.
Described buffer memory divides multiple batches the data that described needs read are returned described request of data end, and described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node.
Step 202: described buffer memory sends response instruction to described request of data end.
Buffer memory sends response instruction to described request of data end, represents the operation that can perform and read data.
Step 203: described in be buffered in described buffer memory search described need read data.
Request of data end reads data velocity ratio from buffer memory reads from memory node wants fast, therefore preferentially search the described data needing to read in the buffer, if had, just by buffer memory directly by this data return data request end.
Step 204: judge whether there are the described data needing to read in described buffer memory, if had, performs step 205; If no, perform step 206 if had.
Step 205: will the data return data request end of reading be needed.
Step 206: described buffer memory, according to the described address information needing the data read, reads from described memory node the data that described in a group, address information is corresponding.
Buffer memory has obtained the address information needing the data read from described read requests instruction, after receiving described read requests instruction, buffer memory reads needing the data read from described memory node with regard to starting immediately, and completes the reading to data corresponding to address information described in first group.
Step 207: data corresponding for address information described in a group are returned described request of data end by described buffer memory.
If described buffer memory one batch return data amount can be more than or equal to the data volume needing the address information of the data read corresponding described in a group, data corresponding for address information described in a group are returned described request of data end.
Described be buffered in data corresponding for address information described in a group are returned described request of data end while, also data corresponding to described address information are organized to next and carry out pre-read, from the operation of described memory node pre-reading data and buffer memory by the operation of the data return data request end of having read points of two streamlines respectively and continue to carry out, operation does not each other interfere with each other buffer memory.
Step 208: judge to need the data read whether all to return, if do not return step 206;
While buffer memory returns to request of data end the data that described in first group, address information is corresponding, return step 206.
Such as: request of data end needs the data of reading 100 pages, buffer memory is divided into 4 batches to return, each batch returns 25 pages, and request of data end sends read requests instruction to buffer memory, and this read requests instruction comprises the address information of the data of the reading 100 pages of needs.Be buffered in after receiving read requests instruction, response instruction is sent to request of data end, search spatial cache and whether have the data needing to read, if do not had, the address information of the data of the reading 100 pages of the needs comprised according to read requests instruction, starts from memory node, read corresponding data.
The data volume that buffer memory once can read data from described memory node be 25 pages, and the address information of 100 page datas is divided into 4 groups by buffer memory, and often group comprises the address information of 25 page datas, and the data volume of its correspondence needs the data volume that reads identical with one batch.Buffer memory reads the data of 25 pages corresponding to a group address message from memory node, and starts these data of 25 pages to return described request of data end.When being buffered in 25 page data returning first batch, continue the data of 25 pages corresponding to reading second group address message simultaneously.After 25 page datas of the first batch return by buffer memory, the data of 25 pages of second group also read and are over, the data of corresponding for the second group address message 25 pages are returned described request of data end by buffer memory, continue the data of 25 pages corresponding to reading the 3rd group address message simultaneously, repeat said process, until the complete return data request end of the data of 100 pages.
The method for reading data that the present embodiment provides, utilize buffer memory buffer memory return data to the time of request of data end, simultaneously to needing the data read to carry out pre-read, pre-buffering task and data return task and perform simultaneously, separate, increase read request hit rate, improve reading rate, save the time of reading from memory node.
The embodiment of the present invention additionally provides a kind of method for reading data, and compared with previous method for reading data, its difference is, if the data volume that the data of batch can return is less than the data volume that described in a group, address information is corresponding,
Buffer memory, by after the digital independent of a group, needs multiple batches just can return these group data to described request of data end, while being buffered in return data to described request of data end, continues to read from memory node the data needing to read.
Such as: request of data end needs the data of reading 100 pages, buffer memory is divided into 4 batches to return, each batch returns 25 pages, and request of data end sends read requests instruction to buffer memory, and this read requests instruction comprises the address information of the data of the reading 100 pages of needs.Be buffered in after receiving read requests instruction, response instruction is sent to request of data end, search spatial cache and whether have the data needing to read, if do not had, the address information of the data of the reading 100 pages of the needs comprised according to read requests instruction, starts from memory node, read corresponding data.
Buffer memory can be once 25 pages from the data volume of described memory node reading data, and the address information of 100 page datas is divided into 4 groups by buffer memory, and often group comprises the address information of 25 page datas, and the data volume of its correspondence is greater than the data volume of a batch of needs reading.Buffer memory reads the data of 25 pages corresponding to a group address message from memory node, and starts these data of 25 pages to return described request of data end, and one batch returns 5 pages, 5 times altogether, completes returning first group of data.When being buffered in 25 page data returning first group, continue the data of 25 pages corresponding to reading second group address message simultaneously.
When buffer memory is by 25 page datas of the first batch, divide after having returned for 5 times, the data of 25 pages of second group also read and are over, buffer memory starts the data of corresponding for the second group address message 25 pages to return described request of data end, namely perform and return operation the 6-10 time, continue the data of 25 pages corresponding to reading the 3rd group address message simultaneously, repeat said process, until the complete return data request end of the data of 100 pages.
The method for reading data that the present embodiment provides, utilize buffer memory buffer memory return data to the time of request of data end, simultaneously to needing the data read to carry out pre-read, pre-buffering task and data return task and perform simultaneously, separate, increase read request hit rate, improve reading rate, save the time of reading from memory node.
See Fig. 4, a kind of data fetch device that the embodiment of the present invention provides, be applied to data reading system, described data reading system comprises request of data end, buffer memory and stores the memory node of data, described data fetch device is arranged at described buffer memory, described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node, and described device comprises:
Read requests instruction reception unit 401, for receiving the read requests instruction that described request of data end sends, described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading.
Response instruction sending unit 402, for sending response instruction to described request of data end, and searches the described data needing to read in described buffer memory.
Storing data-reading unit 403, if for there is no the described data needing to read in described buffer memory, start pre-buffering task, reading data corresponding to described address information according to the described address information of the data read that needs from described memory node;
Described storage data-reading unit 403 also for: read from described memory node the data that described in a group, address information is corresponding.
Data return unit 404, return task for starting data, and the data read from described memory node are returned described request of data end;
Described data return unit 404 also for: data corresponding for address information described in described a group are returned described request of data end;
Described data return unit 404 also for: the data of described batch are returned described request of data end by described buffer memory, repeat this step, until the data that described in a group, address information is corresponding all return described request of data end.
The data fetch device that the embodiment of the present invention provides, its technique effect realizing principle and generation is identical with preceding method embodiment, is concise and to the point description, and the not mentioned part of device embodiment part can with reference to corresponding contents in preceding method embodiment.
See Fig. 1, a kind of data reading system that the embodiment of the present invention provides, described data reading system comprises request of data end 100, buffer memory 200 and stores the memory node 300 of data,
Described request of data end 100 sends read requests instruction to described buffer memory 200;
Described buffer memory 200 receives the read requests instruction that described request of data end 100 sends, and described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Described buffer memory 200 sends response instruction to described request of data end 100, searches the described data needing to read in described buffer memory 200; If do not have the described data needing to read in described buffer memory 200, according to the described address information needing the data read, described buffer memory 200 reads data corresponding to described address information from described memory node 300;
The data read from described memory node 300 are returned described request of data end 100 by described buffer memory 200.
The data reading system that the embodiment of the present invention provides, its technique effect realizing principle and generation is identical with preceding method embodiment, is concise and to the point description, and the not mentioned part of system embodiment part can with reference to corresponding contents in preceding method embodiment.
See Fig. 5, the embodiment of the present invention also provides a kind of data fetch device 600, comprising: processor 500, storer 501, bus 502 and communication interface 503, and described processor 500, communication interface 503 are connected by bus 502 with storer 501; The executable module of processor 500 for storing in execute store 501, such as computer program.
Wherein, storer 501 may comprise high-speed random access memory (RAM:Random Access Memory), also non-labile storer (non-volatile memory) may also be comprised, such as at least one magnetic disk memory.Realize the communication connection between this system network element and at least one other network element by least one communication interface 503 (can be wired or wireless), can internet be used, wide area network, local network, Metropolitan Area Network (MAN) etc.
Bus 502 can be isa bus, pci bus or eisa bus etc.Described bus can be divided into address bus, data bus, control bus etc.For ease of representing, only representing with a four-headed arrow in Fig. 5, but not representing the bus only having a bus or a type.
Wherein, storer 501 is for storage program, and described processor 500, after receiving execution instruction, performs described program, the method performed by device of the flow definition that aforementioned embodiment of the present invention any embodiment discloses can be applied in processor 500, or is realized by processor 500.
Processor 500 may be a kind of integrated circuit (IC) chip, has the processing power of signal.In implementation procedure, each step of said method can be completed by the instruction of the integrated logic circuit of the hardware in processor 500 or software form.Above-mentioned processor 500 can be general processor, comprises central processing unit (Central Processing Unit is called for short request of data end), network processing unit (Network Processor is called for short NP) etc.; Can also be digital signal processor (DSP), special IC (ASIC), ready-made programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components.Can realize or perform disclosed each method, step and the logic diagram in the embodiment of the present invention.The processor etc. of general processor can be microprocessor or this processor also can be any routine.Step in conjunction with the method disclosed in the embodiment of the present invention directly can be presented as that hardware decoding processor is complete, or combines complete by the hardware in decoding processor and software module.Software module can be positioned at random access memory, flash memory, ROM (read-only memory), in the storage medium of this area maturations such as programmable read only memory or electrically erasable programmable storer, register.This storage medium is positioned at storer 501, and processor 500 reads the information in storer 501, completes the step of said method in conjunction with its hardware.
In addition, the process flow diagram in accompanying drawing and block diagram show system according to multiple embodiment of the present invention, the architectural framework in the cards of method and computer program product, function and operation.In this, each square frame in process flow diagram or block diagram can represent a part for module, program segment or a code, and a part for described module, program segment or code comprises one or more executable instruction for realizing the logic function specified.Also it should be noted that at some as in the realization of replacing, the function marked in square frame also can be different from occurring in sequence of marking in accompanying drawing.Such as, in fact two continuous print square frames can perform substantially concurrently, and they also can perform by contrary order sometimes, and this determines according to involved function.Also it should be noted that, the combination of the square frame in each square frame in block diagram and/or process flow diagram and block diagram and/or process flow diagram, can realize by the special hardware based system of the function put rules into practice or action, or can realize with the combination of specialized hardware and computer instruction.
The computer program carrying out a kind of data fetch device that the embodiment of the present invention provides, comprise the computer-readable recording medium storing program code, the instruction that described program code comprises can be used for performing the method described in previous methods embodiment, specific implementation see embodiment of the method, can not repeat them here.
Those skilled in the art can be well understood to, and for convenience and simplicity of description, the specific works process of the system of foregoing description, device and unit, with reference to the corresponding process in preceding method embodiment, can not repeat them here.
In several embodiments that the application provides, should be understood that disclosed system, apparatus and method can realize by another way.Device embodiment described above is only schematic, such as, the division of described unit, be only a kind of logic function to divide, actual can have other dividing mode when realizing, again such as, multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can be ignored, or do not perform.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some communication interfaces, and the indirect coupling of device or unit or communication connection can be electrical, machinery or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, also can be that the independent physics of unit exists, also can two or more unit in a unit integrated.
If described function using the form of SFU software functional unit realize and as independently production marketing or use time, can be stored in a computer read/write memory medium.Based on such understanding, the part of the part that technical scheme of the present invention contributes to prior art in essence in other words or this technical scheme can embody with the form of software product, this computer software product is stored in a storage medium, comprising some instructions in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) perform all or part of step of method described in each embodiment of the present invention.And aforesaid storage medium comprises: USB flash disk, portable hard drive, ROM (read-only memory) (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. various can be program code stored medium.
It should be noted that, in this article, the such as relational terms of first and second grades and so on is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, article or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, article or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment comprising described key element and also there is other identical element.
In describing the invention, also it should be noted that, unless otherwise clearly defined and limited, term " setting ", " installation ", " being connected ", " connection " should be interpreted broadly, and such as, can be fixedly connected with, also can be removably connect, or connect integratedly; Can be mechanical connection, also can be electrical connection; Can be directly be connected, also indirectly can be connected by intermediary, can be the connection of two element internals.For the ordinary skill in the art, concrete condition above-mentioned term concrete meaning in the present invention can be understood.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.It should be noted that: represent similar terms in similar label and letter accompanying drawing below, therefore, once be defined in an a certain Xiang Yi accompanying drawing, then do not need to define further it and explain in accompanying drawing subsequently.
Claims (10)
1. a method for reading data, is characterized in that, is applied to data reading system, and described data reading system comprises request of data end, buffer memory and stores the memory node of data, and described method comprises:
Described buffer memory receives the read requests instruction that described request of data end sends, and described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Send response instruction to described request of data end, in described buffer memory, search the described data needing to read;
If there is no the described data needing to read in described buffer memory, start pre-buffering task, read data corresponding to described address information according to the described address information of the data read that needs from described memory node;
Described buffer memory starts data and returns task, and the data read from described memory node are returned described request of data end;
Described pre-buffering task and described data return task and perform simultaneously, separate.
2. method according to claim 1, is characterized in that,
Describedly read data corresponding to described address information from described memory node, comprising: once read data corresponding to whole described address information from described memory node;
Described the data read from described memory node are returned described request of data end, comprising: data corresponding for the whole described address information read from described memory node are once returned described request of data end.
3. method according to claim 1, it is characterized in that, described buffer memory divides multiple batches the data that described needs read are returned described request of data end, described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node, if described buffer memory one batch return data amount can be more than or equal to the data volume needing the address information of the data read corresponding described in a group
Describedly read data corresponding to described address information from described memory node, comprising:
The data that described in one group, address information is corresponding are read from described memory node;
Described the data read from described memory node are returned described request of data end, comprising:
Data corresponding for address information described in described a group are returned described request of data end by described buffer memory;
Described method also comprises: repeat said process, until the described data read that need all read and return described request of data end by described buffer memory.
4. method according to claim 3, is characterized in that, if the data volume that the data of batch can return is less than the data volume that described in a group, address information is corresponding,
Data corresponding for address information described in described a group are returned described request of data end by described buffer memory, comprising:
The data of described one batch are returned described request of data end by described buffer memory, repeat this step, until the data that described in a group, address information is corresponding all return described request of data end.
5. a data fetch device, is characterized in that, is applied to data reading system, and described data reading system comprises request of data end, buffer memory and stores the memory node of data, and described data fetch device is arranged at described buffer memory, and described device comprises:
Read requests instruction reception unit, for receiving the read requests instruction that described request of data end sends, described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Response instruction sending unit, for sending response instruction to described request of data end, and searches the described data needing to read in described buffer memory;
Storing data-reading unit, if for there is no the described data needing to read in described buffer memory, start pre-buffering task, reading data corresponding to described address information according to the described address information of the data read that needs from described memory node;
Data return unit, return task for starting data, and the data read from described memory node are returned described request of data end.
6. data fetch device according to claim 5, is characterized in that,
Described storage data-reading unit also for: once read data corresponding to whole described address information from described memory node;
Described data return unit also for: data corresponding for the whole described address information read from described memory node are once returned described request of data end.
7. data fetch device according to claim 5, it is characterized in that, described buffer memory divides multiple batches the data that described needs read are returned described request of data end, described buffer memory is divided into many groups according to the address information of the data that all described needs read by the data volume that once can read data from described memory node, if described buffer memory one batch return data amount can be more than or equal to the data volume needing the address information of the data read corresponding described in a group
Described storage data-reading unit also for: read from described memory node the data that described in a group, address information is corresponding;
Described data return unit also for: data corresponding for address information described in described a group are returned described request of data end by described buffer memory.
8. data fetch device according to claim 7, is characterized in that, if the data volume that the data of batch can return is less than the data volume that described in a group, address information is corresponding,
Described data return unit also for: the data of described batch are returned described request of data end by described buffer memory, repeat this step, until the data that described in a group, address information is corresponding all return described request of data end.
9. a data reading system, is characterized in that, described data reading system comprises request of data end, buffer memory and stores the memory node of data,
Described request of data end sends read requests instruction to described buffer memory;
Described buffer memory receives the read requests instruction that described request of data end sends, and described read requests instruction comprises the address information of the quantity information of the data that needs read and the data of described needs reading;
Described buffer memory sends response instruction to described request of data end, searches the described data needing to read in described buffer memory; If do not have the described data needing to read in described buffer memory, according to the described address information needing the data read, described buffer memory reads data corresponding to described address information from described memory node;
The data read from described memory node are returned described request of data end by described buffer memory.
10. data reading system according to claim 9, is characterized in that, described data reading system is applied to distributed memory system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410836741.3A CN104461943B (en) | 2014-12-29 | 2014-12-29 | Method for reading data, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410836741.3A CN104461943B (en) | 2014-12-29 | 2014-12-29 | Method for reading data, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104461943A true CN104461943A (en) | 2015-03-25 |
| CN104461943B CN104461943B (en) | 2017-10-27 |
Family
ID=52908028
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410836741.3A Expired - Fee Related CN104461943B (en) | 2014-12-29 | 2014-12-29 | Method for reading data, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104461943B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104715048A (en) * | 2015-03-26 | 2015-06-17 | 浪潮集团有限公司 | File system caching and pre-reading method |
| CN106598488A (en) * | 2016-11-24 | 2017-04-26 | 北京小米移动软件有限公司 | Distributed data reading method and device |
| CN106802952A (en) * | 2017-01-18 | 2017-06-06 | 广东睿江云计算股份有限公司 | The processing method of mass data, extracting method and processing unit |
| CN107623722A (en) * | 2017-08-21 | 2018-01-23 | 云宏信息科技股份有限公司 | A kind of remote data caching method, electronic equipment and storage medium |
| CN107992271A (en) * | 2017-12-21 | 2018-05-04 | 郑州云海信息技术有限公司 | Data pre-head method, device, equipment and computer-readable recording medium |
| CN111124491A (en) * | 2019-12-12 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Batch processing method, device, equipment and storage medium |
| CN111857817A (en) * | 2019-04-25 | 2020-10-30 | 比亚迪股份有限公司 | Data reading method, data reading device and data reading system |
| CN112131144A (en) * | 2020-09-27 | 2020-12-25 | 深圳市芯天下技术有限公司 | Serial interface NAND memory chip and method for reading data from the same |
| CN114077588A (en) * | 2020-08-20 | 2022-02-22 | 荣耀终端有限公司 | Pre-reading method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103077225A (en) * | 2012-12-31 | 2013-05-01 | 华为技术有限公司 | Data reading method, device and system |
| US20130326026A1 (en) * | 2012-06-01 | 2013-12-05 | Sk Telecom Co., Ltd. | Local caching device, system and method for providing content caching service |
| CN103635887A (en) * | 2013-09-23 | 2014-03-12 | 华为技术有限公司 | Data caching method and storage system |
| CN104158863A (en) * | 2014-08-02 | 2014-11-19 | 成都致云科技有限公司 | Cloud storage mechanism based on transaction-level whole-course high-speed buffer |
-
2014
- 2014-12-29 CN CN201410836741.3A patent/CN104461943B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130326026A1 (en) * | 2012-06-01 | 2013-12-05 | Sk Telecom Co., Ltd. | Local caching device, system and method for providing content caching service |
| CN103077225A (en) * | 2012-12-31 | 2013-05-01 | 华为技术有限公司 | Data reading method, device and system |
| CN103635887A (en) * | 2013-09-23 | 2014-03-12 | 华为技术有限公司 | Data caching method and storage system |
| CN104158863A (en) * | 2014-08-02 | 2014-11-19 | 成都致云科技有限公司 | Cloud storage mechanism based on transaction-level whole-course high-speed buffer |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104715048B (en) * | 2015-03-26 | 2018-06-05 | 浪潮集团有限公司 | A kind of file system cache pre-reading method |
| CN104715048A (en) * | 2015-03-26 | 2015-06-17 | 浪潮集团有限公司 | File system caching and pre-reading method |
| CN106598488A (en) * | 2016-11-24 | 2017-04-26 | 北京小米移动软件有限公司 | Distributed data reading method and device |
| CN106598488B (en) * | 2016-11-24 | 2019-08-13 | 北京小米移动软件有限公司 | Distributed data read method and device |
| CN106802952A (en) * | 2017-01-18 | 2017-06-06 | 广东睿江云计算股份有限公司 | The processing method of mass data, extracting method and processing unit |
| CN107623722A (en) * | 2017-08-21 | 2018-01-23 | 云宏信息科技股份有限公司 | A kind of remote data caching method, electronic equipment and storage medium |
| CN107992271A (en) * | 2017-12-21 | 2018-05-04 | 郑州云海信息技术有限公司 | Data pre-head method, device, equipment and computer-readable recording medium |
| CN111857817A (en) * | 2019-04-25 | 2020-10-30 | 比亚迪股份有限公司 | Data reading method, data reading device and data reading system |
| CN111857817B (en) * | 2019-04-25 | 2024-02-20 | 比亚迪半导体股份有限公司 | Data reading method, data reading device and data reading system |
| CN111124491A (en) * | 2019-12-12 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Batch processing method, device, equipment and storage medium |
| CN111124491B (en) * | 2019-12-12 | 2022-04-22 | 浪潮(北京)电子信息产业有限公司 | Batch processing method, device, equipment and storage medium |
| CN114077588A (en) * | 2020-08-20 | 2022-02-22 | 荣耀终端有限公司 | Pre-reading method and device |
| CN112131144A (en) * | 2020-09-27 | 2020-12-25 | 深圳市芯天下技术有限公司 | Serial interface NAND memory chip and method for reading data from the same |
| CN112131144B (en) * | 2020-09-27 | 2023-09-26 | 芯天下技术股份有限公司 | Serial interface NAND memory chip and method for reading data from same |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104461943B (en) | 2017-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104461943A (en) | Data reading method, device and system | |
| US8725961B2 (en) | Systems, methods, and devices for configuring a device | |
| CN107590099B (en) | Method for accessing slave machines by multiple hosts, extension device and storage medium | |
| US8555156B2 (en) | Inferring that a message has been read | |
| JP2015191655A (en) | Method and apparatus for generating recommendation page | |
| CN102810050A (en) | Log data writing method and log system | |
| CN104571956A (en) | Data writing method and splitting device | |
| CN104462225A (en) | Data reading method, device and system | |
| US7895376B2 (en) | Hardware configuration information system, method, and computer program product | |
| CN110020273B (en) | Method, device and system for generating thermodynamic diagram | |
| EP2919120A1 (en) | Memory monitoring method and related device | |
| US20180276507A1 (en) | Machine learning classifiers | |
| CN111737564A (en) | Information query method, device, equipment and medium | |
| CN104767761A (en) | Cloud storage platform access control method and device | |
| CN113127382A (en) | Data reading method, device, equipment and medium for additional writing | |
| CN104503868A (en) | Data synchronizing method, device and system | |
| CN104199977A (en) | Method for creating information search based on data in database | |
| CN104571957A (en) | Data reading method and assembling device | |
| CN104699788A (en) | Database query method and device | |
| CN104506636A (en) | Method and device for synchronizing data | |
| CN116610262A (en) | Method, device, equipment and medium for reducing SSD sequential reading delay | |
| CN112513822B (en) | Information processing method, device, equipment and system | |
| CN111090633A (en) | Small file aggregation method, device and equipment of distributed file system | |
| CN104536800A (en) | Virtual machine placement method and device | |
| CN116166575B (en) | Method, device, equipment, medium and program product for configuring access segment length |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20171027 Termination date: 20191229 |