CN107728930A - A kind of data access method and system - Google Patents
A kind of data access method and system Download PDFInfo
- Publication number
- CN107728930A CN107728930A CN201610652036.7A CN201610652036A CN107728930A CN 107728930 A CN107728930 A CN 107728930A CN 201610652036 A CN201610652036 A CN 201610652036A CN 107728930 A CN107728930 A CN 107728930A
- Authority
- CN
- China
- Prior art keywords
- data
- storage space
- written
- read
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000003860 storage Methods 0.000 claims description 450
- 230000003068 static effect Effects 0.000 claims description 25
- 238000005096 rolling process Methods 0.000 claims description 10
- 238000013500 data storage Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 239000002699 waste material Substances 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of data access method, methods described includes：When to determine data to be written be cold data, the first memory space locking of the cold data will be stored, and the cold data is write into first memory space；Data storage to be read is inquired when first memory space, the data to be read are removed from first memory space, and the data to be read are write into second memory space or threeth memory space corresponding with the data type of the data to be read, read the data to be read from second memory space or the 3rd memory space.The present invention further simultaneously discloses a kind of data access arrangement.
Description
Technical Field
The invention relates to a data processing technology, in particular to a data access method and a data access system.
Background
Cloud computing constitutes a brand-new internet technology Service providing mode, and the industry generally divides the Service providing mode into three levels of Infrastructure As A Service (IAAS), Platform As A Service (PAAS), and Software As A Service (SAAS), wherein the IAAS mainly provides data computing and data storage services, and in a large-scale storage environment, a server system generally divides data into three types, namely static data, dynamic data, and cold data, but in the prior art, the cold data is generally stored together with the static data or the dynamic data, which causes waste of storage resources in the server system and network blocking.
Disclosure of Invention
In order to solve the existing technical problem, embodiments of the present invention are expected to provide a data access method and system, which can reduce the waste of storage resources of a server system.
The technical scheme of the embodiment of the invention is realized as follows:
according to an aspect of an embodiment of the present invention, there is provided a data access method, including:
when data to be written is determined to be cold data, locking a first storage space for storing the cold data, and writing the cold data into the first storage space;
when the data to be read is inquired and stored in the first storage space, the data to be read is shifted out of the first storage space, the data to be read is written into a second storage space or a third storage space corresponding to the data type of the data to be read, and the data to be read is read from the second storage space or the third storage space.
In the foregoing scheme, the determining that the data to be written is cold data includes:
and when detecting that the data to be written carries prompt information of cold data, determining that the data to be written is the cold data.
In the above scheme, after the writing the cold data into the first storage space, the method further includes:
when the cold data is successfully written, unlocking the first storage space, and updating the space information of the first storage space;
and when the cold data writing fails, unlocking the first storage space, and rolling back the space information of the first storage space.
In the above scheme, the method further comprises:
and when determining that the data to be written is non-cold data and a storage space for storing the data to be written is recorded in the metadata information, locking a second storage space or a third storage space corresponding to the data type of the data to be written, and writing the data to be written into the second storage space or the third storage space.
In the above scheme, after the writing the data to be written into the second storage space or the third storage space, the method further includes:
when the data to be written is successfully written, unlocking the second storage space or the third storage space, and updating the space information of the second storage space or the third storage space;
and when the data to be written fails to be written, unlocking the first storage space or the second storage space, and rolling back the space information of the second storage space or the third storage space.
In the above scheme, when the data to be read is not stored in the first storage space, the data to be read is read from the second storage space or the third storage space according to the data type of the data to be read.
In the above scheme, the first storage space is used for storing cold data;
the second storage space is used for storing dynamic data;
and the third storage space is used for storing static data.
According to another aspect of the embodiments of the present invention, there is provided a data access system, including:
the device comprises a data writing module, a metadata information module, a data reading module, a first storage space, a second storage space and a third storage space; wherein,
the data writing module is used for locking a first storage space for storing cold data and writing the cold data into the first storage space when the data to be written is determined to be the cold data;
the metadata information module is used for storing the space information of the first storage space, the second storage space and the third storage space and the data attributes of the user data stored in the first storage space, the second storage space and the third storage space;
the data reading module is used for shifting the data to be read out from the first storage space by the data writing module when the metadata information module inquires that the data to be read out is stored in the first storage space, writing the data to be read into a second storage space or a third storage space corresponding to the data type of the data to be read out, and reading the data to be read out from the second storage space or the third storage space.
In the foregoing scheme, the data writing module is specifically configured to determine that the data to be written is cold data when detecting that the data to be written carries prompt information of the cold data.
In the above scheme, the data writing module is further configured to unlock the first storage space and update the space information of the first storage space when the cold data is successfully written; and when the cold data writing fails, unlocking the first storage space, and rolling back the space information of the first storage space.
In the above scheme, the data writing module is further configured to determine that data to be written is non-cold data, and lock a second storage space or a third storage space corresponding to a data type of the data to be written when a storage space for storing the data to be written is recorded in the metadata information module, and write the data to be written in the second storage space or the third storage space.
In the above scheme, the data writing module is further configured to unlock the second storage space or the third storage space when the data to be written is written successfully, and update space information of the second storage space or the third storage space; and when the data to be written fails to be written, unlocking the second storage space or the third storage space, and rolling back the space information of the second storage space or the third storage space.
In the above scheme, the data reading module is further configured to, when it is found in the metadata information module that the data to be read is not stored in the first storage space, read the data to be read from the second storage space or the third storage space according to the data type of the data to be read.
In the above scheme, the first storage space is used for storing cold data;
the second storage space is used for storing dynamic data;
and the third storage space is used for storing static data.
According to the data access method and the data access system provided by the embodiment of the invention, when the data to be written is determined to be cold data, a first storage space for storing the cold data is locked, and the cold data is written into the first storage space; when the data to be read is inquired and stored in the first storage space, the data to be read is shifted out of the first storage space, the data to be read is written into a second storage space or a third storage space corresponding to the data type of the data to be read, and the data to be read is read from the second storage space or the third storage space. Therefore, user data which is necessary to be stored but not used for a long time can be stored into a separate storage space as cold data, and the storage space for storing static data or dynamic data can be saved, so that the space waste in a server system is reduced, the network blockage is reduced, and the utilization rate of the storage space is improved.
Drawings
FIG. 1 is a flow chart of a data access method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data access system according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
FIG. 1 is a flow chart of a data access method according to an embodiment of the present invention; as shown in fig. 1, the method includes:
step 101, when determining that data to be written is cold data, locking a first storage space for storing the cold data, and writing the cold data into the first storage space;
generally, in a server system, data types of user data are divided into static data, dynamic data and cold data, wherein the static data refers to data which does not need to interact with a server database, and is created at a client when a user views the data, for example, data such as pictures, audio and video files and the like which do not need to be changed; the dynamic data refers to data which can obtain information in a database through a code of a server and can directly reflect a transaction process, and when a user views the data, the data can change along with time change, such as the daily access volume of system data and database data; cold data refers to data that is necessary to be stored but is not used often, for example, dynamic data and/or static data that has not been used for more than a preset storage time; the server system is divided into a first storage space, a second storage space and a third storage space in the physical storage device according to the different types of data, wherein the first storage space is mainly used for storing cold data; the second storage space is mainly used for storing dynamic data; the third storage space is mainly used for storing static data; the first storage space, the second storage space and the third storage space collectively form a heterogeneous storage pool in the physical storage device.
After receiving a data service request, a server system firstly judges whether the data service request is a data writing request or a data reading request; when the server system determines that the data service request is a data writing request, judging the data type of data to be written in the data writing request, and finding a storage space for storing the data type in metadata information in the server system according to the data type; here, the data service request may be sent by the user side, or may be sent by the metadata information module in the server system. When the server system determines that the data type of the data to be written is cold data according to the data to be written in the data writing request, locking the first storage space for storing the cold data, wherein the locked first storage space cannot be used by other clients any more; when the server system locks the first storage space, inquiring a storage space for currently storing the data to be written from the server system, and when the server system inquires that the data to be written is currently stored in the second storage space, moving the data to be written out of the second storage space and writing the data to be written into the first storage space; and when the server inquires that the data to be written is currently stored in the third storage space, the data to be written is moved out of the third storage space and written into the first storage space. When the data to be written is successfully written into the first storage space from the second storage space or the third storage space, unlocking the first storage space, and updating the space information of the first storage space, where updating the first storage space information mainly refers to updating the space allowance and the space usage capacity of the first storage space, and may also update the space information of the second storage space or the third storage space that historically stores the cold data. When the data to be written fails to be written into the first storage space from the second storage space or the third storage space, the server system still unlocks the first storage space, and rolls back the space information of the first storage space, and meanwhile, can also roll back the space information of the second storage space or the third storage space which historically stores the cold data. The rolling back here refers to restoring the space information of the first storage space and/or the second storage space and the third storage space to the state of storing data last time.
In this embodiment of the present invention, the determining that the data to be written is cold data includes:
and when determining that the data writing request carries prompt information of cold data, determining that the data to be written is the cold data.
Here, space information of the first storage space, the second storage space, and the third storage space, and data attributes of user data stored in each storage space are recorded in a metadata information module in the server system, where the space information mainly refers to total capacity of the storage space, current storage space usage capacity, remaining capacity of the storage space, and the like; the data attributes mainly refer to the size, storage location, storage time and the like of data. When the storage time of the user data stored in the second storage space and/or the third storage space reaches a preset storage time, a metadata information module in the server system sends a data service request to the server system, after the server system receives the data service request, it is firstly determined that the data service request is a data writing request, then it is determined whether the data writing request carries prompt information of cold data, and when it is determined that the data writing request carries prompt information of the cold data, it is determined that the data to be written is the cold data. Here, the prompt information includes: and prompting that the user data stored in the second storage space or the third storage space exceeds a preset storage time. For example, the preset storage time is 6 months, when the storage time of the user data stored in the second storage space and/or the third storage space reaches the preset storage time of 6 months, the data write request received by the server system will carry prompt information marked as cold data, and determine that the data to be written is the cold data according to the prompt information, and then move the data to be written out of the corresponding second storage space or the third storage space and write the data into the first storage space. The specific preset storage time can be set according to the self-requirement, and the specific time is limited.
102, when inquiring that data to be read is stored in the first storage space, moving the data to be read out from the first storage space, writing the data to be read into a second storage space or a third storage space corresponding to the data type of the data to be read, and reading the data to be read from the second storage space or the third storage space;
when the server system determines that the data service request is a data reading request, firstly, the data type of the data to be read is judged, then, the storage position of the data to be read is inquired from the server system, and when the data to be read is inquired in the server system and stored in the first storage space, the data to be read is removed from the first storage space, and the data to be read is written into a second storage space or a third storage space corresponding to the data type of the data to be read. Specifically, when the data to be read is dynamic data, the data to be read is written into the second storage space after being moved out of the first storage space; and when the data to be read is static data, the data to be read is written into the third storage space after being shifted out from the first storage space. Then, the server system reads the data to be read from the second storage space or the third storage space through a storage driving interface; when the server system inquires that the data to be read is not stored in the first storage space but is stored in the second storage space or the third storage space, the data to be read is directly read from the second storage space or the third storage space through the storage driving interface.
In the embodiment of the invention, when the data to be written is determined to be not cold data and the storage space for storing the data to be written is recorded in the server system, the second storage space or the third storage space corresponding to the data type of the data to be written is locked, and the data to be written is written into the second storage space or the third storage space.
Here, when the server system determines that the data type of the data to be written is non-cold data, first determining the data type of the data to be written, then querying, in the server system, whether the storage space of the second storage space and/or the third storage space corresponding to the data type is sufficient for storing the data to be written, and when it is queried in metadata information of the server system that there is a sufficient storage space for storing the data to be written in the second storage space or the third storage space, locking the second storage space or the third storage space corresponding to the data type of the data to be written. Specifically, when the data type of the data to be written is dynamic data, whether the dynamic data is external data written into the second storage space or backup storage data written into the second storage space is further determined; when the data type of the data to be written is static data, further judging whether the static data is external data written into a third storage space or backup storage data written into the third storage space; when the server system determines that the dynamic data or the static data is external data written into a second storage space or external data written into a third storage space, directly writing the data to be written into the second storage space or the third storage space through a storage driving interface; and when the server system determines that the dynamic data or the static data is backup storage data written into a second storage space or backup storage data written into a third storage space, the data to be written is moved out from the first storage space through the storage drive interface and is written into the second storage space or the third storage space. When the data to be written is successfully written into the second storage space or the third storage space, unlocking the second storage space or the third storage space, and updating the space information of the second storage space or the third storage space; and when the data to be written fails to be written, the server system still unlocks the second storage space or the third storage space, and rolls back the space information of the second storage space or the third storage space.
Fig. 2 is a schematic structural diagram of a data access system according to an embodiment of the present invention, as shown in fig. 2, the system includes: a data writing module 201, a metadata information module 202, a data reading module 203, a first storage space 204, a second storage space 205 and a third storage space 206; wherein,
the data writing module 201 is configured to lock a first storage space 204 for storing cold data when it is determined that data to be written is cold data, and write the cold data into the first storage space 204;
the metadata information module 202 is configured to store space information of the first storage space 204, the second storage space 205, and the third storage space 206, and data attributes of user data stored in the first storage space 204, the second storage space 205, and the third storage space 206;
the data reading module 203 is configured to, when the metadata information module 202 queries that data to be read is stored in the first storage space 204, move the data to be read out of the first storage space 204 by the data writing module 201, write the data to be read into the second storage space 205 and the third storage space 206 corresponding to the data type of the data to be read, and read the data to be read from the second storage space 205 and the third storage space 206.
Here, the data access system may be a server system, and generally, in the data access system, data types of user data are divided into static data, dynamic data, and cold data, where the static data refers to data that does not need to interact with a server database, and is data that has been created at a client when a user views the data, for example, data that does not need to be changed, such as a picture, an audio/video file, and the like, and the static data is stored in the third storage space 206; the dynamic data refers to data which can obtain information in a database through a code of a server and can directly reflect a transaction process, and when a user views the data, the data can change along with time change, for example, the daily access volume of system data and database data, and the dynamic data is stored in the second storage space 205; the cold data refers to data that is necessary to be stored but is not frequently used, for example, dynamic data and/or static data that is not used for more than a preset storage time, and the cold data is stored in the first storage space 204; the first storage space 204, the second storage space 205, and the third storage space 206 collectively form a heterogeneous storage pool in the physical storage device, where the first storage space 204 may be a backup storage pool in the heterogeneous storage pool, the second storage space 205 may be a block storage pool in the heterogeneous storage pool, and the third storage space 206 may be an object storage pool in the heterogeneous storage pool.
When a data access system receives a data service request, firstly, judging whether the data service request is a data writing request or a data reading request; when the data access system determines that the data service request is a data write-in request, the data service request is sent to the data write-in module 201, the data write-in module 201 determines a data type of data to be written in the data service request, and a storage space for storing the data type is found in the metadata information module 202 according to the data type; here, the data service request may be sent by a user side, or may be sent by the metadata information module 202. When the data writing module 201 determines that the data type is cold data according to data to be written in the data writing request, the first storage space 204 for storing the cold data is locked, and the locked first storage space 204 cannot be used by other clients any more; after the data writing module 201 locks the first storage space 204, querying a storage space in which the data to be written is currently stored from the metadata information module 202, and when the data to be written is queried to be currently stored in the second storage space 205, removing the data to be written from the second storage space 205 through a storage drive interface and writing the data to be written into the first storage space 204; when the data writing module 201 queries that the data to be written is currently stored in the third storage space 206, the data to be written is removed from the third storage space 206 through a storage driver interface and is written into the first storage space 204. Here, the data access system further includes a storage driver module 207, and the storage driver interface may be located in the storage driver module 207 in the data access system. When the data to be written is successfully written into the first storage space 204, the data writing module 201 unlocks the first storage space 204 and updates the space information of the first storage space 204, where updating the space information of the first storage space 204 mainly refers to updating the space margin, the space usage capacity, and the like of the first storage space 204, and may also update the space information of the second storage space 205 or the third storage space 206 that historically stores the cold data. When the data to be written fails to be written, the data writing module 201 still unlocks the first storage space 204, and rolls back the space information of the first storage space 204, and may also roll back the space information of the second storage space 205 or the third storage space 206 where the history stores the cold data. Here, the rolling back means that the space information of the first storage space 204 and/or the second storage space 205 or the third storage space 206 is restored to a state of storing data last time.
In this embodiment of the present invention, the data writing module 201 is specifically configured to determine that the data to be written is cold data when detecting that the data to be written carries prompt information of the cold data.
Here, the metadata information module 202 records space information of the first storage space 204, the second storage space 205, and the third storage space 206, and data attributes of user data stored in each storage space, where the space information mainly refers to total capacity of the storage space, current storage space usage capacity, remaining storage space capacity, and the like; the data attributes mainly refer to the size, storage location, storage time and the like of data. When the metadata information module 202 determines that the storage time of the user data stored in the second storage space 205 and/or the third storage space 206 reaches a preset time, the metadata information module 202 sends a data service request to the data writing module 201. After receiving the data service request, the data writing module 201 first determines that the data service request is a data writing request, then determines whether the data writing request carries a prompt message of cold data, and determines that the data to be written is the cold data when determining that the data writing request carries the prompt message of cold data. For example, the preset storage time is 6 months, when the metadata information module 202 determines that the storage time of the user data stored in the second storage space 205 and/or the third storage space 206 has reached the preset storage time of 6 months, the data write request received by the data write module 201 will carry a prompt message marked as cold data, and determine that the data to be written is cold data according to the prompt message, and then remove the cold data from the corresponding second storage space 205 or the third storage space 206 and write the cold data into the first storage space 204. The specific preset storage time can be set according to the self-requirement, and the specific time is limited.
In this embodiment of the present invention, the data writing module 201 is further configured to determine that data to be written is not cold data, and lock the second storage space 205 or the third storage space 206 corresponding to the data type of the data to be written when a storage space for storing the data to be written is recorded in the metadata information module 202, and write the data to be written in the second storage space 205 or the third storage space 206.
Here, when the data writing module 201 determines that the data type of the data to be written is non-cold data, it is queried in the metadata information module 202 whether the storage space of the second storage space 205 and/or the third storage space 206 is sufficient to store the data to be written, and when it is queried in the metadata information module 202 that there is sufficient storage space in the second storage space 205 and/or the third storage space 206 to store the data to be written, the second storage space 205 and/or the third storage space 206 corresponding to the data type of the data to be written is locked. Specifically, when the data type of the data to be written is dynamic data, it is further determined whether the dynamic data is external data written in the second storage space 205 or backup storage data written in the second storage space 205; when the data type of the data to be written is static data, further determining whether the static data is external data written in the third storage space 206 or backup storage data written in the third storage space 206; when the server system determines that the dynamic data or the static data is external data written into the second storage space 205 or external data written into the third storage space 206, the data to be written is directly written into the second storage space 205 or the third storage space 206 from the data writing module 201 through a storage driver interface. When the data writing module 201 determines that the dynamic data or the static data is backup storage data written in the second storage space 205 or backup storage data written in the third storage space 206, the data to be written is removed from the first storage space 204 through the storage drive interface and written in the second storage space 205 or the third storage space 206. When the data writing module 201 successfully writes the data to be written into the second storage space 205 or the third storage space 206, unlocking the second storage space 205 or the third storage space 206, and updating the space information of the second storage space 205 or the third storage space 206; when the data to be written fails to be written, the data writing module 201 still unlocks the second storage space 205 or the third storage space 206, and rolls back the space information of the second storage space 205 or the third storage space 206.
In the embodiment of the present invention, when the data access system determines that the data service request is a data read request, the data read request is sent to the data read module 203, the data read module 203 first determines a data type of the data to be read, then queries a storage location of the data to be read from the metadata information module 202 according to the determined data type of the data to be read, when querying that the data to be read is stored in the first storage space 204 in the metadata information module 202, the metadata information module 202 sends a data write request to the data write module 201, after receiving the data write request sent by the metadata information module 202, the data write module 201 removes the data to be read from the first storage space 204 according to the data write request, and writes the data into the second storage space 205 or the third storage space 206 corresponding to the data to be read, specifically, when the data to be read is dynamic data, the data to be read is moved out of the first storage space 204 and then written into the second storage space 205; when the data to be read is static data, the data to be read is moved out of the first storage space 204 and then written into the third storage space 206. Then, the data to be read is read from the second storage space 205 or the third storage space 206 by the data reading module 203 through a storage drive interface in the storage drive module 207; when the data reading module 203 queries in the metadata information module 202 that the data to be read is not stored in the first storage space 204 but is stored in the second storage space 205 or the third storage space 206, the data to be read is directly read from the second storage space 205 or the third storage space 206 through a storage driver interface in the storage driver module 207.
In practical applications, the data writing module 201, the metadata information module 202, the data reading module 203, the first storage space 204, the second storage space 205, the third storage space 206, and the storage driving module 207 may be implemented by a Central Processing Unit (CPU), a microprocessor unit (MPU), a Digital Signal Processor (DSP), or a Field Programmable Gate Array (FPGA), etc. located in a server system.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, systems according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (14)
1. A method for accessing data, the method comprising:
when data to be written is determined to be cold data, locking a first storage space for storing the cold data, and writing the cold data into the first storage space;
when the data to be read is inquired and stored in the first storage space, the data to be read is shifted out of the first storage space, the data to be read is written into a second storage space or a third storage space corresponding to the data type of the data to be read, and the data to be read is read from the second storage space or the third storage space.
2. The method of claim 1, wherein the determining that the data to be written is cold data comprises:
and when detecting that the data to be written carries prompt information of cold data, determining that the data to be written is the cold data.
3. The method of claim 1, wherein after said writing said cold data to said first storage space, said method further comprises:
when the cold data is successfully written, unlocking the first storage space, and updating the space information of the first storage space;
and when the cold data writing fails, unlocking the first storage space, and rolling back the space information of the first storage space.
4. The method of claim 1, further comprising:
and when determining that the data to be written is non-cold data and a storage space for storing the data to be written is recorded in the metadata information, locking a second storage space or a third storage space corresponding to the data type of the data to be written, and writing the data to be written into the second storage space or the third storage space.
5. The method of claim 4, wherein after the writing the data to be written to the second storage space or the third storage space, the method further comprises:
when the data to be written is successfully written, unlocking the second storage space or the third storage space, and updating the space information of the second storage space or the third storage space;
and when the data to be written fails to be written, unlocking the first storage space or the second storage space, and rolling back the space information of the second storage space or the third storage space.
6. The method of claim 1, wherein when the data to be read is not stored in the first memory space, the data to be read is read from the second memory space or the third memory space according to a data type of the data to be read.
7. The method according to any one of claims 1-6, wherein the first storage space is for storing cold data;
the second storage space is used for storing dynamic data;
and the third storage space is used for storing static data.
8. A data access system, the system comprising:
the device comprises a data writing module, a metadata information module, a data reading module, a first storage space, a second storage space and a third storage space; wherein,
the data writing module is used for locking a first storage space for storing cold data and writing the cold data into the first storage space when the data to be written is determined to be the cold data;
the metadata information module is used for storing the space information of the first storage space, the second storage space and the third storage space and the data attributes of the user data stored in the first storage space, the second storage space and the third storage space;
the data reading module is used for shifting the data to be read out from the first storage space by the data writing module when the metadata information module inquires that the data to be read out is stored in the first storage space, writing the data to be read into a second storage space or a third storage space corresponding to the data type of the data to be read out, and reading the data to be read out from the second storage space or the third storage space.
9. The system according to claim 8, wherein the data writing module is specifically configured to determine that the data to be written is cold data when detecting that the data to be written carries prompt information of cold data.
10. The system of claim 8, wherein the data writing module is further configured to unlock the first storage space and update the space information of the first storage space when the cold data writing is successful; and when the cold data writing fails, unlocking the first storage space, and rolling back the space information of the first storage space.
11. The system according to claim 8, wherein the data writing module is further configured to determine that the data to be written is non-cold data, and when a storage space for storing the data to be written is recorded in the metadata information module, lock a second storage space or a third storage space corresponding to a data type of the data to be written, and write the data to be written into the second storage space or the third storage space.
12. The system according to claim 11, wherein the data writing module is further configured to unlock the second storage space or the third storage space when the data to be written is successfully written, and update space information of the second storage space or the third storage space; and when the data to be written fails to be written, unlocking the second storage space or the third storage space, and rolling back the space information of the second storage space or the third storage space.
13. The system according to claim 8, wherein the data reading module is further configured to, when it is queried in the metadata information module that the data to be read is not stored in the first storage space, read the data to be read from the second storage space or the third storage space according to a data type of the data to be read.
14. The system according to any one of claims 8-13, wherein the first storage space is configured to store cold data;
the second storage space is used for storing dynamic data;
and the third storage space is used for storing static data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610652036.7A CN107728930A (en) | 2016-08-10 | 2016-08-10 | A kind of data access method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610652036.7A CN107728930A (en) | 2016-08-10 | 2016-08-10 | A kind of data access method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107728930A true CN107728930A (en) | 2018-02-23 |
Family
ID=61199432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610652036.7A Pending CN107728930A (en) | 2016-08-10 | 2016-08-10 | A kind of data access method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107728930A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101217571A (en) * | 2008-01-18 | 2008-07-09 | 清华大学 | Write/read document operation method applied in multi-copy data grid system |
| US20120209893A1 (en) * | 2011-02-15 | 2012-08-16 | Samsung Electronics Co., Ltd. | File system operating method and devices using the same |
| CN103499951A (en) * | 2013-09-13 | 2014-01-08 | 中国石油天然气股份有限公司 | Dynamic balance adjusting system of oil pumping unit based on Web framework |
| CN103688259A (en) * | 2011-05-19 | 2014-03-26 | 甲骨文国际公司 | Techniques for automatic data placement with compression and columnar storage |
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
| CN105224240A (en) * | 2014-05-30 | 2016-01-06 | 北京忆恒创源科技有限公司 | By the method for data write storage device, device and memory device |
-
2016
- 2016-08-10 CN CN201610652036.7A patent/CN107728930A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101217571A (en) * | 2008-01-18 | 2008-07-09 | 清华大学 | Write/read document operation method applied in multi-copy data grid system |
| US20120209893A1 (en) * | 2011-02-15 | 2012-08-16 | Samsung Electronics Co., Ltd. | File system operating method and devices using the same |
| CN103688259A (en) * | 2011-05-19 | 2014-03-26 | 甲骨文国际公司 | Techniques for automatic data placement with compression and columnar storage |
| CN103499951A (en) * | 2013-09-13 | 2014-01-08 | 中国石油天然气股份有限公司 | Dynamic balance adjusting system of oil pumping unit based on Web framework |
| CN105224240A (en) * | 2014-05-30 | 2016-01-06 | 北京忆恒创源科技有限公司 | By the method for data write storage device, device and memory device |
| CN104850358A (en) * | 2015-05-26 | 2015-08-19 | 华中科技大学 | Magnetic-optical-electric hybrid storage system and data acquisition and storage method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR20190044145A (en) | Processing mutations for a remote database | |
| CN104580439B (en) | Method for uniformly distributing data in cloud storage system | |
| CN110275876B (en) | Method and device for laterally expanding database | |
| CN103034592B (en) | Data processing method and device | |
| CN105573859A (en) | Data recovery method and device of database | |
| WO2018006587A1 (en) | File storage method, terminal, and storage medium | |
| CN111522502B (en) | Data deduplication method and device, electronic equipment and computer-readable storage medium | |
| US10423580B2 (en) | Storage and compression of an aggregation file | |
| CN113806143B (en) | Remote disaster recovery method, system and electronic equipment | |
| CN115114232A (en) | Method, device and medium for enumerating historical version objects | |
| CN109271097B (en) | Data processing method, data processing device and server | |
| CN114721594A (en) | Distributed storage method, device, equipment and machine readable storage medium | |
| CN114647658A (en) | Data retrieval method, device, equipment and machine-readable storage medium | |
| US9424362B2 (en) | Storing and publishing contents of a content store | |
| CN103838782A (en) | Method and device for achieving file scanning and storage | |
| CN106528876B (en) | The information processing method and distributed information processing system of distributed system | |
| CN107728930A (en) | A kind of data access method and system | |
| CN109669623B (en) | File management method, file management device, electronic equipment and storage medium | |
| CN109558249B (en) | Control method and device for concurrent operation | |
| CN108345431B (en) | Data reading method and device | |
| CN110543575B (en) | Media file retrieval method and device of vehicle-mounted external storage equipment | |
| CN108255848B (en) | Method, equipment and system for realizing condition updating | |
| CN111782590A (en) | File reading method and device | |
| CN111371818A (en) | Data request verification method, device and equipment | |
| US8965857B2 (en) | Method and system to avoid space bloating during run-time compression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180223 |