CN108345431A - A kind of method and device of digital independent - Google Patents
A kind of method and device of digital independent Download PDFInfo
- Publication number
- CN108345431A CN108345431A CN201711490140.1A CN201711490140A CN108345431A CN 108345431 A CN108345431 A CN 108345431A CN 201711490140 A CN201711490140 A CN 201711490140A CN 108345431 A CN108345431 A CN 108345431A
- Authority
- CN
- China
- Prior art keywords
- data
- read
- acquisition system
- data acquisition
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000004590 computer program Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of method and device of digital independent, this method include access service layer receive in reading the data client request reading data acquisition system start resettlement notification message when, the location information of remaining data and record for identifying the data read that finally succeed in data acquisition system in data acquisition system is read in pause, when receiving the notice that data acquisition system resettlement is completed, according to the mark of data acquisition system, it determines the new storage location of data acquisition system, continues to read the remaining data being located at after the data that last success is read in new storage location.Continue to read the remaining data after being located at the data that last success is read from the new storage location of data acquisition system by the mark of the data read further according to last success after the completion of the data acquisition system resettlement that client request is read, it can to return to the data that all requests are read in single reading is according to request, without unsuccessfully leading to re-initiate the case where single reading is according to request because reading.
Description
Technical field
The present embodiments relate to technical field of memory more particularly to a kind of method and devices of digital independent.
Background technology
At present in object storage service system, metadata is used to describe some attributes of the data of storage, such as describes
The information such as the storage location of the data of storage, the size for storing data.The data of storage are typically separate storage with metadata,
It can be directed to respectively different features in this way and select different storage mediums and way to manage, to accelerate to read and enumerate constant speed
Degree.For example, the size of data is determined by user, for system in addition to limiting certain range, size is simultaneously uncontrollable；And first number
According to being generated by system, size and structure can be self-defined, it might even be possible to neatly select structuring, relevant database
To store.
Object is written into object storage system, (object includes data to be stored and is determined according to the data
Metadata) when, be generally divided into two processes, first preserve data to be stored, further according to some features of data to be stored
It generates a metadata and preserves.
When reading stored object, flow is just the opposite, usually first reads the metadata of the object of storage, root
After obtaining the essential informations such as the storage location of data according to the metadata of reading, then the corresponding storage location from bottom storage system
On read out data and return to user.
In order to improve the utilization rate of bottom storage system, it will usually the multiple data to be stored for including by a file
Interspersed to be stored on different memory spaces, i.e. the multiple data stored on a memory space may belong to different numbers
According to.In this way, when user asks to delete a file, it is likely to result on some memory space and a large amount of discrete skies occurs
White storage region is not continuous since these free memory area domains are too discrete, can not accommodate a larger storage pair again
As, if it is possible to remaining valid data block on this memory space is moved to another new white space, then before
Memory space on can form a large amount of continuous empty storage space, can continue to accommodate new data.
As shown in Figure 1, storing data 1 on some memory space arrives data 5, user has initiated to ask to delete later
Data 2, data 4, can cause the white space for occurring discrete on this memory space.Under normal circumstances, these discrete skies
White space is difficult to be utilized again, when new storage object is written again, it is impossible to which whether memory space before rescaning
There is suitable white space, this does not allow in time.Therefore, it generally requires and carries out data-moving processing, i.e., it will be effective
Data 1, data 3 and data 5 move on new memory space, after data are moved, so that it may to vacate one piece of blank sky
Between continue to accommodate new data.
After completing the data resettlement of bottom storage system, correspondingly need to update the corresponding first number of allochthonous data
According to the storage location of the data after being moved with record.If before more new metadata or during more new metadata, user
Read data request is initiated, it is likely that the scene of reading failure due to can not find data occur.As shown in Fig. 2, dotted arrow table
The data resettlement task shown, such as an object to be read are made of 3 data.User has initiated to read asking for data at this time
It asks, after obtaining the storage location of primary data 1, data 2 and data 3 by the old metadata of reading, starts to read data 1, and will
The data 1 of reading return to user.When attempting to read data 2, data 1, data 2 and data 3 are from old memory space at this time
New memory space is moved to, and old memory space is recovered in the system, system will be considered that loss of data exception, even if being deposited originally
Storage space, which is set, ceaselessly to be retried, and after retrying repeatedly, the information for reading failure finally can be also returned to user.
After reading failure, needs user to re-initiate a completely new read data request, can just read new member
Data, and then data 1, data 2 and data 3 could be re-read in the memory space to after moving according to new metadata, this
Kind of situation is not suitable for the system for having high reliability request, the data resettlement more frequent foreground especially in bottom storage system
Have under a large amount of scenes concurrently read, reading mortality but will significantly improve.
Invention content
The embodiment of the present invention provides a kind of method and device of digital independent, can solve data resettlement and read number with user
According to the problem of reading fails under concurrent scene.
In a first aspect, providing a kind of method of digital independent, access service layer receives client in reading the data
When the data acquisition system that end request is read starts the notification message of resettlement, pause is read remaining data in the data acquisition system and is remembered
The location information for the data read that finally succeed in the data acquisition system in mark is employed, the data acquisition system includes at least one
A data, each data correspond to a mark, and the data acquisition system resettlement is to remove the data acquisition system from a memory space
Another memory space is moved on to, when receiving the notice that the data acquisition system resettlement is completed, according to the mark of the data acquisition system
Know, determine the new storage location of the data acquisition system, then according to record for identify in the data acquisition system finally at
The location information for the data that work(is read, the data acquisition system after the resettlement stored in the new storage location relay fetch bit of resuming studies
Remaining data after the data that the last success is read.
By in reading the data, if the data acquisition system that access service layer receives client request reading starts to remove
When the notification message moved, starts pause and read remaining data in the data acquisition system and record for identifying in the data set
The mark for the data read that finally succeed in conjunction is completed logical in the data acquisition system resettlement that subsequently received client request is read
Continue to read positioned at most from the new storage location of data acquisition system further according to the finally mark of the successful data read after knowing message
Remaining data after the data read that succeed afterwards can to return to the number that all requests are read in single reading is according to request
According to without unsuccessfully leading to re-initiate the case where single reading is according to request because reading.
In a kind of possible design, the position for identify the data for the reading that finally succeeds in the data acquisition system
Information can be at least one of following information：Finally succeed the corresponding data mark of data read in the data acquisition system
Know, succeeded the corresponding Data Identification of each data read in the data acquisition system, in the data acquisition system most
The corresponding Data Identification of next adjacent data for the data read that succeed afterwards.
By succeeding to the end for identifying the location information for the data read that finally succeed in data acquisition system and can obtain
The position of the data of reading thereby may be ensured that continue to read from the new storage location of data acquisition system to be located at and finally successfully read
Data after remaining data.
In a kind of possible design, the access service layer can according to the mark of the data acquisition system, read with it is described
The corresponding new metadata of mark of data acquisition system determines that the data acquisition system is new and deposits according to the new metadata read
Storage space is set.
The corresponding new metadata of the mark is read by the mark of data acquisition system, the new storage of data acquisition system can be obtained
Position, so as to continue to read the remaining data not read in new storage location, without data Layer always
It repeats to read the case where not reading data failure.
In a kind of possible design, the access service layer pause is read remaining data in the data acquisition system and is recorded
Before the mark of the data read that finally succeeds in the data acquisition system, the access service layer can be by obtaining the visitor
The read data request that family end is sent, the read data request includes the mark for the data acquisition system that the request is read, according to institute
The mark for the data acquisition system that request is read is stated, the metadata for the data acquisition system that the request is read is read, determines that the request is read
The storage location of the storage location of the data acquisition system taken, the data acquisition system read according to the request reads what the request was read
At least one data in data acquisition system are established input and output (input/output, IO) stream with the client, will be read
At least one data the client is sent in the form of data flow.
In a kind of possible design, the access service layer remaining data and is remembered in the data acquisition system is read in pause
After the mark for recording the data read that finally succeed in the data acquisition system, the access service layer can be kept and the visitor
The IO that family end is established flows, and continues in the data acquisition system after the resettlement that the access service layer stores in the new storage location
Reading is after the remaining data after the data that the last success is read, so that it may will continue the remaining data read
The IO that access is kept with the client flows, and sends the remaining data for continuing to read to the client.
It, can be surplus by what is read by keeping flowing with the IO that client is established, and after data acquisition system resettlement is completed
The case where remainder is flowed according to the IO for accessing the holding again, and the state that data can be kept to transmit always is not in interruption.
Second aspect provides a kind of digital independent device, including：I/O interface and processor；The I/O interface, user receive
With transmission data and Data Identification；The processor, in reading the data, determining that the I/O interface receives client
When the data acquisition system that end request is read starts the notification message of resettlement, pause is read remaining data in the data acquisition system and is remembered
The location information for the data read that finally succeed in the data acquisition system in mark is employed, the data acquisition system includes at least one
A data, each data correspond to a mark, and the data acquisition system resettlement is to remove the data acquisition system from a memory space
Move on to another memory space；And when determining that the I/O interface receives the notice that the data acquisition system resettlement is completed, root
According to the mark of the data acquisition system, the new storage location of the data acquisition system is determined；And according to record for identifying described
The location information for the data read that finally succeed in data acquisition system, the data after the resettlement stored in the new storage location
Aggregate relay resume studies fetch bit in it is described it is last success read data after remaining data.
In a kind of possible design, the position for identify the data for the reading that finally succeeds in the data acquisition system
Information is at least one of following information：
Finally succeed the corresponding Data Identification of data read in the data acquisition system；
Succeeded the corresponding Data Identification of each data read in the data acquisition system；
The corresponding Data Identification of next adjacent data for the data read that finally succeed in the data acquisition system.
In a kind of possible design, the processor determines that the data acquisition system is new according to the mark of the data acquisition system
Storage location when, corresponding with the mark of the data acquisition system new member can be read according to the mark of the data acquisition system
Data；According to the new metadata read, the new storage location of the data acquisition system is determined.
In a kind of possible design, the processor remaining data and records use in the data acquisition system is read in pause
Before the location information for identifying the data read that finally succeed in the data acquisition system, it can also be obtained by the I/O interface
The read data request for taking the client to send, the read data request include the mark for the data acquisition system that the request is read
Know；According to the mark for the data acquisition system that the request is read, the metadata for the data acquisition system that the request is read is read, determines institute
State the storage location for the data acquisition system that request is read；The storage location of the data acquisition system read according to the request is asked described in reading
Seek at least one data in the data acquisition system of reading；And the streams of the IO between the I/O interface and the client are established, it will read
At least one data got are sent to the client in the form of data flow.
In a kind of possible design, the processor remaining data and records use in the data acquisition system is read in pause
It after the location information for identifying the data read that in the data acquisition system finally succeed, can also continue to that the IO is kept to connect
The IO established between mouth and the client flows；Data after the resettlement that the processor stores in the new storage location
Aggregate relay is resumed studies after remaining data of the fetch bit after the data that the last success is read, so that it may will continue to read
Remaining data access kept between the I/O interface and the client IO stream, to send the residue for continuing to read
Data give the client.
The third aspect provides a kind of device of digital independent, including：Processor and memory；The memory is stored with meter
Calculation machine executes instruction, which is connect with the memory by the bus, and when the equipment is run, which executes this and deposit
Computer executed instructions of reservoir storage, so that the equipment executes the method as described in above-mentioned first aspect any one.
Fourth aspect provides a kind of computer readable storage medium, including computer-readable instruction, when computer is read simultaneously
When executing the computer-readable instruction so that computer executes the method as described in first aspect any one.
5th aspect, provides a kind of computer program product, including computer-readable instruction, when computer is read and is executed
The computer-readable instruction so that computer executes the method as described in first aspect any one.
Description of the drawings
Fig. 1 is a kind of schematic diagram of data resettlement in the prior art；
Fig. 2 is a kind of schematic diagram of digital independent in the prior art；
Fig. 3 is a kind of structural schematic diagram of system architecture provided in an embodiment of the present invention；
Fig. 4 is a kind of flow diagram of the method for digital independent provided in an embodiment of the present invention；
Fig. 5 is a kind of flow diagram of the method for digital independent provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of digital independent device provided in an embodiment of the present invention；
Fig. 7 is a kind of structural schematic diagram of digital independent device provided in an embodiment of the present invention.
Specific implementation mode
Fig. 3 shows that a kind of system architecture that the embodiment of the present invention is applicable in, the system architecture include client 301 and deposit
Store up server 302.Wherein, client 301 is communicated with storage server 302, for asking to read to storage server 302
With write-in data.
The storage server 302 may include access service layer, metadata layer and data Layer；Access service layer is used for and visitor
Family end 301 interacts, and receives the request message that client 301 is sent, and execute operation corresponding with request message.First number
According to the corresponding metadata of data stored in layer for storing data layer, metadata is mainly to describe the information of data attribute, is used
To support such as to indicate storage location, storage size function.Data Layer is used to store the data of client request storage.
Currently, data are pronounced has correspondence to ensure its reliability for a most basic service in each layer of system
Reliability Assurance have and retry accordingly if access service layer is after detecting exception, to solve unstable networks, flash
Or server is busy, and corresponding retry mechanism is also had in bottom data layer, with ensure will not because of some disk failure and
It causes entirely to ask failure.
But before data Layer each layer frequency converter, be essentially all to be confined to each layer simply to be retried, example
Such as, in the retry mechanism of access service layer just come into force in the initial of service foundation；And data Layer for memory space
The reading of the data of middle storage is then to attempt ceaselessly to read from the same memory space, the same position always.In this way in data
Resettlement more frequently and is read under the concurrent scene of data manipulation, and digital independent mortality significantly improves.If the number that user reads
, can be because the reading of certain one piece of data fails when larger according to itself, the IO streams between server and user can be interrupted, before at
The data of work(transmission also can all be cancelled, and the significant wastage to Internet resources and time is caused.
To solve the above-mentioned problems, Fig. 4 illustratively shows a kind of side of digital independent provided in an embodiment of the present invention
The flow of method, the flow can be executed by access service layer, which can be located in storage server.
Step 401, access service layer receives the data acquisition system that client request is read and starts in reading the data
When the notification message of resettlement, pause read data acquisition system in remaining data and record for identify in data acquisition system finally at
The location information for the data that work(is read.
In embodiments of the present invention, the data acquisition system that client request is read may include at least one data, per number
According to a corresponding mark, for example, data acquisition system 1 include data 1, data 2 ..., data n, wherein n is positive integer.Data n is
The corresponding Data Identification of nth data.The mark of data acquisition system and the Data Identification of data use in embodiments of the present invention
It is digital form, is only example effect, in specific application, can be used for distinguishing different data and difference using others
The mark of data acquisition system does not limit this.Data acquisition system resettlement can be understood as removing data acquisition system from a memory space
Move on to the process of another memory space.Access service layer can encounter data Layer and carry out data resettlement in reading the data
Task, the data acquisition system that can receive server background transmission at this time start the notification message of resettlement, that is to say, that are being read
There are the tasks that the data stored in data Layer are moved while the task for evidence of fetching, it may also be said to be the number read
It is being moved according to set.Access service layer can suspend remaining data in the data acquisition system for reading and currently reading at this time,
And record the location information for identifying the data read that finally succeed in the data acquisition system.This is used to identify in data set
The location information for the data read that finally succeed in conjunction can be at least one of following information：It is last in the data acquisition system
The corresponding Data Identification of data that success is read；Succeeded the corresponding data mark of each data read in data acquisition system
Know；The corresponding Data Identification of next adjacent data for the data read that finally succeed in data acquisition system.That is, here may be used
The data of first data to be read in Data Identification or remaining data to record the data that last success is read
Mark, can also be the Data Identification of each data for the reading that succeeded.For example, having data 1, data 2, number in data acquisition system
According to four data such as 3 and data 4.After having read data 1 and data 2, start to receive the data acquisition system quilt when reading data 3
In integral moving to another memory space, access service layer can record the Data Identification of data 2, can also record data 1
With the Data Identification of data 2, next adjacent, Data Identification of data 3 for not reading with data 2 can also be recorded.
In this case, client request can be read the task progress of the data in data acquisition system by access service layer
Pause is further continued for being read out the task of data after waiting the task execution that data acquisition system is moved.
Access service layer is generally asked by first obtaining the reading data of client transmission before data task is read in pause
It asks, which includes the mark for the data acquisition system that request is read.The data acquisition system that the request is read includes at least one
A data can read the data acquisition system of request reading by the mark for the data acquisition system that the request is read from metadata layer
The corresponding metadata of mark, can be determined from the metadata read the request read data acquisition system storage location
(storage location before moving), that is, the data acquisition system is in the old storage location of data Layer.The metadata is service
The information that attribute is closed for descriptor data set that device will generate after the data storage to data Layer in data acquisition system.Access service
Layer is after the storage location for determining data acquisition system that request is read, so that it may with the data acquisition system read to data Layer request
Storage location gets on to read at least one data in the data acquisition system that the request is read.Then, the meeting of access service layer and client
IO streams are established, at least one data read are sent to client in the form of data flow.
Access service layer is while remaining data during the data acquisition system currently read is read in above-mentioned pause, access
Service layer can also continue to keep flowing with the IO that client is established, to prevent IO streams from interrupting.Keep the IO established with client
Stream can make user the case where client not can be appreciated that digital independent failure, will not see that reading pause, user can be seen
It is being transmitted always to data, there is no interrupting, so as to promote user experience.
Step 402, access service layer is when receiving the notice that data acquisition system resettlement is completed, according to the mark of data acquisition system
Know, determines the new storage location of data acquisition system.
When the data acquisition system that the client request in data Layer is read, which is moved, to be completed, server background can send out one and remove
The notice of completion is moved, access service layer is when receiving the notice that data acquisition system resettlement is completed, after the completion of data acquisition system resettlement,
A new metadata can be generated, which is stored in metadata layer.It is moved and is completed due to the data in data acquisition system
After can more new metadata, continue read data task when, need the data read may be just not in storage before
On position, remaining data in data acquisition system can not be read in storage location old before, therefore just need basis should
The mark of data acquisition system reads the data acquisition system to metadata layer and moves the new metadata generated later, the new metadata
What is indicated is the new storage location of the data acquisition system.After reading new metadata corresponding with the mark of the data acquisition system,
It can obtain the new storage location of the data acquisition system.
Step 403, access service layer is used to identify the data read that in data acquisition system finally succeed according to record
Location information, the data acquisition system relaying after the resettlement stored in new storage location resume studies fetch bit in the number of last success reading
According to remaining data later.
Access service layer can be used for after obtaining the new storage location of data acquisition system according to what is recorded in step 401
The location information for identifying the data read that finally succeed in data acquisition system, after the resettlement stored in the new storage location
Data acquisition system relays remaining data of the fetch bit after data that the last success is read of resuming studies.Finally succeed the data read
When location information is data that last success is read corresponding Data Identification, the number of the data directly read according to the last success
According to mark, the corresponding data of Data Identification for the data that the last success is read first are found in new storage location, are then read
Remaining data of the fetch bit after the data that the last success is read.It is to have become in the location information for the data that last success is read
When the corresponding Data Identification of each data that work(is read, the last one in each data for reading of first determining to have succeeded
The corresponding Data Identification of data that success is read, the Data Identification of the data then read further according to the last success, new
The corresponding data of Data Identification that the data that the last success is read are found in storage location, last read positioned at the last success
Remaining data after the data of reading.It is the data that last success is read in the location information for the data that last success is read
When the corresponding Data Identification of next adjacent data, the next adjacent data pair for the data that the last success is read can be directly read
The data answered and remaining data later.For example, a data acquisition system includes 5 data, respectively data 1, data 2, number
According to 3, data 4, data 5, for access service layer when data task is read in pause, the last data successfully read of record are number
According to 2, that is to say, that data 1 and data 2 have all been successfully read, and data 3, data 4 and data 5 are in data acquisition system
Remaining data.Access service layer is looked for after obtaining new storage location according to new metadata in new storage location
To data 2, then proceed to read the data 3 being located at after data 2, until reading data 5.
Access service layer can will continue the remaining data access read and be protected with client after continuing to read data
The IO streams held, and the remaining data for continuing to read are sent to client.It is phase the case where user plane is seen in this way
Than the read data request in no data are moved when, only speed is slightly slow, but can succeed in single reading is according to request
All digital independent tasks are completed, are not in the case where reading failure.
In order to clearly explain the flow of digital independent provided in an embodiment of the present invention, will specifically implement below
The flow of the digital independent is described under scene.
As shown in figure 5, the flow specifically includes：
Step 501, client initiates read data request.
Client is initiated to read request of data, includes the mark of data acquisition system in the read data request.The data acquisition system
Be identified as 1, that is, data acquisition system 1.The data acquisition system includes 5 data, respectively data 1, data 2, data 3, data 4
With data 5, wherein number 1,2,3,4 and 5 is respectively the corresponding Data Identification of each data.
Step 502, the metadata information of the data acquisition system is read.
After the read data request for receiving client transmission, the data acquisition system that reading is identified as 1 corresponds to access service layer
Metadata information.
Step 503, the location information of the data acquisition system on a data layer is obtained.
Access service layer can get the number after reading the corresponding metadata information of the data acquisition system for being identified as 1
According to the storage location of set on a data layer.
Step 504, it navigates to position on a data layer and starts to read data.
Access service layer navigates to data Layer after the storage location for obtaining the data acquisition system for being identified as 1, in data Layer
Start to read the data in the data acquisition system in the storage location of the upper data acquisition system for being identified as 1, be read since data 1 are successively
It takes, as soon as reading successfully data, records the Data Identification of the successful data of the reading.
Step 505, the data of reading are returned.
After the data in reading data acquisition system, data Layer returns to the data read to access service layer.
For example, after reading data 1, data 1 are returned into access service layer, are continuing to read data 2.
Step 506, data are returned to client in the form of IO flows.
The data 1 successfully read received are returned to client by access service layer in the form of IO flows.
Step 507, resettlement task starts, and the data of notification data set start to move.
Access service layer starts to execute the data for being identified as 1 in data Layer during reading data, after server
The resettlement task of set, and the data of the data acquisition system for being identified as 1 currently read to access service layer notice start to remove
It moves.
Step 508, suspend current read request flow of task, record the current last mark for successfully reading data, protect
The IO established with client is held to flow.
The pause of access service layer is currently executing reading data task, that is, pause read in the data acquisition system it is remaining not
The data of reading, and the Data Identification of the data of the current last reading that succeeded is recorded, such as the last successful data read
Data Identification be 2, that is, read data 2, do not read data 3 also, kept the good IO streams established with client.
Step 509, more new metadata, notification data are moved task and are completed.
After data resettlement of the server background in data acquisition system is completed, the corresponding first number of the data acquisition system can be updated
According to, and after having updated metadata, notify the data of the data acquisition system to move task and complete to access service layer.
Step 510, the metadata information of the data acquisition system is re-read.
Access service layer arrives the reading of metadata layer again after the notice that the data resettlement task for receiving data acquisition system is completed
This is taken to be identified as the 1 corresponding metadata information of data acquisition system.
Step 511, the storage location of the data acquisition system on a data layer is obtained.
Access service layer is carrying out necessary school after the metadata for obtaining the data acquisition system for being identified as 1 to metadata
After testing, so that it may be identified as the new storage location of 1 data acquisition system on a data layer to reacquire this.
Step 512, it is repositioned onto the new storage location in data Layer, continues to read remaining number in data acquisition system
According to.
Access service layer is again fixed after the new storage location according to the data acquisition system for being identified as 1 of reacquisition
It then reads and is located at after data 2 in the new storage location of the data acquisition system for being identified as 1 in data Layer in data Layer in position
Remaining data.Before formal read, reading position is first moved forward to the position of data 3 from data 1, it is then true again
This stored in positive slave data Layer is identified as in 1 data acquisition system and reads remaining data, that is, reads data 3, data 4
With data 5.
Step 513, the data of reading are returned.
After reading remaining data in the new storage location of data Layer, return to what this read to access service layer
Remaining data.
Step 514, the IO streams before connecting pause continue to transmit remaining data.
Access service layer is after receiving the remaining data read, before the remaining data newly read are connected
The IO of pause kept with client flows, and continues to transmit the remaining data read.So that in terms of user plane
It is that compared to reading when not moving, only speed is slightly slow, but is successfully completed within a read request to situation
All digital independent tasks, there is no failures.
Based on the same technical idea, Fig. 6 shows a kind of knot of digital independent device 600 provided in an embodiment of the present invention
Structure, the device 600 can be access service layer, can execute the flow of above-mentioned digital independent.
As shown in fig. 6, the device 600 specifically includes：I/O interface 601 and processor 602；
The I/O interface 601, for sending and receiving data and Data Identification；
The processor 602, in reading the data, determining that the I/O interface 601 receives client request
When the data acquisition system of reading starts the notification message of resettlement, remaining data in the data acquisition system are read in pause and record is used for
The location information for the data read that finally succeed in the data acquisition system is identified, the data acquisition system includes at least one number
Correspond to a mark according to, each data, the data acquisition system resettlement for by the data acquisition system from a memory space move to
Another memory space；And when determining that the I/O interface 601 receives the notice that the data acquisition system resettlement is completed, according to
The mark of the data acquisition system determines the new storage location of the data acquisition system；And according to record for identifying in the number
According to the location information for the data read that finally succeed in set, the data set after the resettlement stored in the new storage location
It closes and relays remaining data of the fetch bit after the data that the last success is read of resuming studies.
In a kind of possible design, the processor 602 determines the data acquisition system according to the mark of the data acquisition system
When new storage location, has and be used for：
According to the mark of the data acquisition system, new metadata corresponding with the mark of the data acquisition system is read；
According to the new metadata read, the new storage location of the data acquisition system is determined.
In a kind of possible design, the processor 602 remaining data and is remembered in the data acquisition system is read in pause
It employs before the location information for identifying the data read that finally succeed in the data acquisition system, is additionally operable to：
The read data request that the client is sent is obtained by the I/O interface 601, the read data request includes
The mark for the data acquisition system that the request is read；
According to the mark for the data acquisition system that the request is read, the metadata for the data acquisition system that the request is read is read,
Determine the storage location for the data acquisition system that the request is read；
The storage location of the data acquisition system read according to the request is read in the data acquisition system that the request is read at least
One data；
The IO streams between the I/O interface 601 and the client are established, by least one data read with number
It is sent to the client according to the form of stream.
In a kind of possible design, the processor 602 remaining data and is remembered in the data acquisition system is read in pause
It employs after the location information for identifying the data read that finally succeed in the data acquisition system, is additionally operable to：
The IO established between the I/O interface 601 and the client is kept to flow；
After the resettlement that the processor 602 stores in the new storage location data acquisition system relaying resume studies fetch bit in
After remaining data after the data that the last success is read, it is additionally operable to：
The remaining data for continuing to read is accessed to the IO streams kept between the I/O interface 601 and the client, hair
Send the remaining data read that continues to the client.
Based on the same technical idea, the embodiment of the present invention additionally provides a kind of digital independent device 700, as shown in fig. 7,
The device 700 may include：I/O interfaces 701, processor 702 and memory 703.Processor 702 is used for the behaviour of control device 700
Make；Memory 703 may include read-only memory and random access memory, be stored with the instruction that processor 702 can execute
And data.The a part of of memory 703 can also include non-volatile row random access memory (NVRAM).I/O interfaces 701, place
Reason each component such as device 702 and memory 703 is connected by bus 709, and wherein bus 709 may be used also in addition to including data/address bus
To include power bus, controlling bus and status signal bus in addition.But for the sake of clear explanation, in figure all by various buses
It is designated as bus 709.
A kind of method for reading data that the embodiment of the present invention discloses can be applied in processor 702, or by processor
702 realize.During realization, each step of process flow can pass through the integrated logic circuit of the hardware in processor 702
Or the instruction of software form is completed.Processor 702 can be general processor, digital signal processor, application-specific integrated circuit,
Field programmable gate array either other programmable logic device, discrete gate or transistor logic, discrete hardware components,
It may be implemented or execute disclosed each method, step and the logic diagram in the embodiment of the present invention.General processor can be
Microprocessor or any conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present invention, can directly embody
Execute completion for hardware processor, or in processor hardware and software module combination execute completion.Software module can be with
Positioned at random access memory, flash memory, read-only memory, programmable read only memory or electrically erasable programmable memory, deposit
In the storage medium of this fields such as device maturation.The storage medium is located at memory 703, and processor 702 reads in memory 703 and deposits
The information of storage, in conjunction with the step of a kind of its hardware digital independent of completion.
A kind of method for digital independent that the embodiment of the present invention discloses can be applied in processor 702, or by handling
Device 702 is realized.
Processor 702 is for reading code in memory 703, for executing digital independent in above method embodiment
Flow.
Based on the same technical idea, the embodiment of the present invention also provides a kind of computer readable storage medium, including calculates
Machine readable instruction, when computer is read and executes the computer-readable instruction so that computer executes above-mentioned digital independent
Method.
Based on the same technical idea, the embodiment of the present invention also provides a kind of computer program product, including computer can
Reading instruction, when computer is read and executes the computer-readable instruction so that the method that computer executes above-mentioned digital independent.
Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, depends on the specific application and design constraint of technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
Scope of the present application.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of division of logic function, formula that in actual implementation, there may be another division manner, such as multiple units or component
It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be the indirect coupling by some interfaces, device or unit
It closes or communicates to connect, can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple
In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in each embodiment of the application can be integrated in a processing unit, it can also
It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer read/write memory medium.Based on this understanding, the technical solution of the application is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be expressed in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of step.
And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic disc or CD.
The above, the only specific implementation mode of the application, but the protection domain of the application is not limited thereto, it is any
Those familiar with the art can easily think of the change or the replacement in the technical scope that the application discloses, and should all contain
It covers within the protection domain of the application.Therefore, the protection domain of the application should be based on the protection scope of the described claims.
Claims (12)
1. a kind of method of digital independent, which is characterized in that including：
Access service layer receives the notice that the data acquisition system that client request is read starts to move and disappears in reading the data
When breath, remaining data in the data acquisition system are read in pause and record is finally successfully read for identifying in the data acquisition system
The location information of the data taken, the data acquisition system include at least one data, and each data correspond to a mark, the data
Set resettlement is to move the data acquisition system to another memory space from a memory space；
The access service layer is when receiving the notice that the data acquisition system resettlement is completed, according to the mark of the data acquisition system
Know, determines the new storage location of the data acquisition system；
The access service layer is used to identify in the data acquisition system the finally position of the successful data read according to record
Information, the data acquisition system after the resettlement stored in the new storage location relay fetch bit of resuming studies and are finally successfully read in described
Data after remaining data.
2. the method as described in claim 1, which is characterized in that described finally successfully to be read in the data acquisition system for identifying
The location information of the data taken is at least one of following information：
Finally succeed the corresponding Data Identification of data read in the data acquisition system；
Succeeded the corresponding Data Identification of each data read in the data acquisition system；
The corresponding Data Identification of next adjacent data for the data read that finally succeed in the data acquisition system.
3. the method as described in claim 1, which is characterized in that the access service layer according to the mark of the data acquisition system,
Determine the new storage location of the data acquisition system, including：
The access service layer reads new member corresponding with the mark of the data acquisition system according to the mark of the data acquisition system
Data；
The access service layer determines the new storage location of the data acquisition system according to the new metadata read.
4. the method as described in claim 1, which is characterized in that the access service layer is read in pause in the data acquisition system
Remaining data simultaneously record before being used to identify in the data acquisition system the finally location information of the successful data read, also wrap
It includes：
The access service layer obtains the read data request that the client is sent, and the read data request includes the request
The mark of the data acquisition system of reading；
The mark for the data acquisition system that the access service layer is read according to the request reads the data acquisition system that the request is read
Metadata, determine it is described request read data acquisition system storage location；
The storage location for the data acquisition system that the access service layer is read according to the request reads the data that the request is read
At least one data in set；
The access service layer is established input and output IO with the client and is flowed, by least one data read with data
The form of stream is sent to the client.
5. method as claimed in claim 4, which is characterized in that the access service layer is read in pause in the data acquisition system
After the location information of remaining data and record for identifying the data read that finally succeed in the data acquisition system, also wrap
It includes：
The access service layer keeps flowing with the IO that the client is established；
Data acquisition system relaying after the resettlement that the access service layer stores in the new storage location resumes studies fetch bit in institute
After stating the remaining data after the data that last success is read, further include：
The access service layer will continue the IO that the remaining data read access is kept with the client and flow, send it is described after
The remaining data got resume studies to the client.
6. a kind of digital independent device, which is characterized in that including：Input and output I/O interface and processor；
The I/O interface, for sending and receiving data and Data Identification；
The processor, in reading the data, determining that the I/O interface receives the data of client request reading
When set starts the notification message of resettlement, pause is read remaining data in the data acquisition system and is recorded for identifying described
The location information for the data read that finally succeed in data acquisition system, the data acquisition system includes at least one data, each data
A corresponding mark, the data acquisition system resettlement are empty to another storage to be moved from a memory space data acquisition system
Between；And when determining that the I/O interface receives the notice that the data acquisition system resettlement is completed, according to the data acquisition system
Mark, determines the new storage location of the data acquisition system；And according to record for identify in the data acquisition system finally at
The location information for the data that work(is read, the data acquisition system after the resettlement stored in the new storage location relay fetch bit of resuming studies
Remaining data after the data that the last success is read.
7. device as claimed in claim 6, which is characterized in that described finally successfully to be read in the data acquisition system for identifying
The location information of the data taken is at least one of following information：
Finally succeed the corresponding Data Identification of data read in the data acquisition system；
Succeeded the corresponding Data Identification of each data read in the data acquisition system；
The corresponding Data Identification of next adjacent data for the data read that finally succeed in the data acquisition system.
8. device as claimed in claim 6, which is characterized in that the processor is determined according to the mark of the data acquisition system
When the new storage location of the data acquisition system, has and be used for：
According to the mark of the data acquisition system, new metadata corresponding with the mark of the data acquisition system is read；
According to the new metadata read, the new storage location of the data acquisition system is determined.
9. device as claimed in claim 6, which is characterized in that the processor is remaining in the data acquisition system is read in pause
Data and be recorded in the data acquisition system before the mark for the data read that finally succeed, be additionally operable to：
The read data request that the client is sent is obtained by the I/O interface, the read data request includes the request
The mark of the data acquisition system of reading；
According to the mark for the data acquisition system that the request is read, the metadata for the data acquisition system that the request is read is read, is determined
The storage location for the data acquisition system that the request is read；
The storage location of the data acquisition system read according to the request reads at least one in the data acquisition system that the request is read
Data；
The IO streams between the I/O interface and the client are established, by least one data read with the shape of data flow
Formula is sent to the client.
10. device as claimed in claim 9, which is characterized in that the processor remains in the data acquisition system is read in pause
Remaining data are simultaneously recorded in after the mark for the data read that finally succeed in the data acquisition system, are additionally operable to：
The IO established between the I/O interface and the client is kept to flow；
After the resettlement that the processor stores in the new storage location data acquisition system relaying resume studies fetch bit in it is described most
After remaining data after the data read that succeed afterwards, it is additionally operable to：
The remaining data for continuing to read is accessed to the IO streams kept between the I/O interface and the client, send it is described after
The remaining data got resume studies to the client.
11. a kind of computer readable storage medium, which is characterized in that including computer-readable instruction, when computer is read and is held
When the row computer-readable instruction so that computer executes the method as described in claim 1-5 any one.
12. a kind of computer program product, which is characterized in that including computer-readable instruction, when computer is read and executes institute
State computer-readable instruction so that computer executes the method as described in claim 1-5 any one.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711490140.1A CN108345431B (en) | 2017-12-29 | 2017-12-29 | Data reading method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711490140.1A CN108345431B (en) | 2017-12-29 | 2017-12-29 | Data reading method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108345431A true CN108345431A (en) | 2018-07-31 |
| CN108345431B CN108345431B (en) | 2021-06-22 |
Family
ID=62963402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711490140.1A Active CN108345431B (en) | 2017-12-29 | 2017-12-29 | Data reading method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108345431B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114063935A (en) * | 2022-01-17 | 2022-02-18 | 阿里云计算有限公司 | Method and device for processing data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853229A (en) * | 2010-05-17 | 2010-10-06 | 华为终端有限公司 | Method and device for data transportation, and method of data reading operation and data writing operation |
| CN103914406A (en) * | 2014-03-31 | 2014-07-09 | 中国科学院微电子研究所 | Migration method and migration system for hybrid memories |
| CN106575271A (en) * | 2014-06-23 | 2017-04-19 | 谷歌公司 | Managing storage devices |
| CN106649132A (en) * | 2016-12-29 | 2017-05-10 | 记忆科技(深圳)有限公司 | Solid-state drive junk recovery method |
-
2017
- 2017-12-29 CN CN201711490140.1A patent/CN108345431B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853229A (en) * | 2010-05-17 | 2010-10-06 | 华为终端有限公司 | Method and device for data transportation, and method of data reading operation and data writing operation |
| CN103914406A (en) * | 2014-03-31 | 2014-07-09 | 中国科学院微电子研究所 | Migration method and migration system for hybrid memories |
| CN106575271A (en) * | 2014-06-23 | 2017-04-19 | 谷歌公司 | Managing storage devices |
| CN106649132A (en) * | 2016-12-29 | 2017-05-10 | 记忆科技(深圳)有限公司 | Solid-state drive junk recovery method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114063935A (en) * | 2022-01-17 | 2022-02-18 | 阿里云计算有限公司 | Method and device for processing data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108345431B (en) | 2021-06-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9632826B2 (en) | Prioritizing deferred tasks in pending task queue based on creation timestamp | |
| JPH11119923A (en) | Data access controller and record medium recorded with data access control program | |
| US8131671B2 (en) | Uninterrupted data access during the migration of data between physical file systems | |
| JP2013509658A (en) | Allocation of storage memory based on future usage estimates | |
| CN106126374B (en) | Method for writing data, method for reading data and device | |
| CN106446159B (en) | A kind of method of storage file, the first virtual machine and name node | |
| CN109428913B (en) | Storage capacity expansion method and device | |
| US4523275A (en) | Cache/disk subsystem with floating entry | |
| JPS5943774B2 (en) | Peripheral subsystem control method | |
| CN113495889B (en) | Distributed object storage method and device, electronic equipment and storage medium | |
| CN106133707A (en) | Cache management | |
| CN109996117B (en) | Method and device for acquiring streaming media playing time | |
| CN107329704A (en) | One kind caching mirror method and controller | |
| CN109213447A (en) | A kind of SAN storage volume carry, discharging method and system | |
| JP2015161999A (en) | Storage device, control apparatus, and control program | |
| US10318196B1 (en) | Stateless storage system controller in a direct flash storage system | |
| CN104484291A (en) | Method and device for building equipment mapping | |
| EP0947920B1 (en) | Computer system and method of modifying program in the computer system | |
| CN110147296A (en) | Data processing method, device, equipment and readable storage medium storing program for executing | |
| CN104657088B (en) | A kind of acquisition methods and device of hard disk bad block message | |
| CN108345431A (en) | A kind of method and device of digital independent | |
| CN109032762B (en) | Virtual machine backtracking method and related equipment | |
| US10409748B2 (en) | Bridge device with DMA data transferring to upstream device with reset and clear feature command | |
| US20220027097A1 (en) | Recording apparatus, recording method, storage medium, and memory card | |
| US9317467B2 (en) | Session key associated with communication path |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200415 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Applicant after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 301, A building, room 3, building 301, foreshore Road, No. 310052, Binjiang District, Zhejiang, Hangzhou Applicant before: Hangzhou Huawei Digital Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220216 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after: Huawei Cloud Computing Technologies Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221214 Address after: 518129 Huawei Headquarters Office Building 101, Wankecheng Community, Bantian Street, Gangqu District, Shenzhen, Guangdong Patentee after: Huawei Cloud Computing Technologies Co.,Ltd. Address before: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee before: Huawei Cloud Computing Technologies Co.,Ltd. |
|
| TR01 | Transfer of patent right |