WO2020098654A1 - 基于云存储的数据存储方法、装置和存储介质 - Google Patents

基于云存储的数据存储方法、装置和存储介质 Download PDF

Info

Publication number
WO2020098654A1
WO2020098654A1 PCT/CN2019/117570 CN2019117570W WO2020098654A1 WO 2020098654 A1 WO2020098654 A1 WO 2020098654A1 CN 2019117570 W CN2019117570 W CN 2019117570W WO 2020098654 A1 WO2020098654 A1 WO 2020098654A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
storage
read
solid
identifier
Prior art date
Application number
PCT/CN2019/117570
Other languages
English (en)
French (fr)
Inventor
李丹旺
Original Assignee
杭州海康威视系统技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视系统技术有限公司 filed Critical 杭州海康威视系统技术有限公司
Publication of WO2020098654A1 publication Critical patent/WO2020098654A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656Data buffering arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present application relates to the field of cloud storage technology, and in particular, to a data storage method, device, and storage medium based on cloud storage.
  • Cloud storage has gradually emerged with its massive storage space, high reliability, high concurrency, high scalability and high cost performance. Therefore, more and more users choose to store data in cloud servers.
  • cloud servers generally use mechanical hard disks to store data.
  • the cloud server receives the data to be stored, it stores the data to be stored in the mechanical hard disk, and then refreshes the data in the mechanical hard disk to the physical hard disk.
  • This application provides a cloud storage-based data storage method, device, and storage medium, which can solve the problem that the cloud server cannot support the user's demand for data to be stored and retrieved.
  • the technical solution is as follows:
  • the embodiments of the present application provide a cloud storage-based data storage method.
  • the method is applied to a cache server.
  • the method includes:
  • the periodically dumping the data to be dumped in the first solid state disk to the cloud server includes:
  • the selecting the data to be dumped from the first solid state drive includes:
  • the storage time of the data in the first solid state drive select the data with the furthest storage time away from the current time as the data to be transferred from the first solid state drive; or,
  • the data with the smallest thermal value is selected from the first solid state drive as the data to be dumped, and the thermal value of the data is used to indicate the probability of the data being read.
  • the method further include:
  • the method further includes :
  • the method further includes:
  • the reading the data to be read according to the data identifier of the data to be read includes:
  • the first storage of the data to be read is determined from the first storage record according to the data identifier of the data to be read Path, according to the first storage path of the data to be read, reading the data to be read from the first solid state drive;
  • the second identifier of the data to be read is determined from the second storage record according to the data identifier of the data to be read
  • the storage path reads the data to be read from the cloud server according to the second storage path of the data to be read.
  • the method before determining the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, the method further includes:
  • the third storage record contains the data identifier of the data to be read, and the third storage record is a storage record of the backup server of the cache server;
  • the third storage of the data to be read is determined from the third storage record according to the data identification of the data to be read Path, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server;
  • an embodiment of the present application provides a data storage device based on cloud storage.
  • the device is applied to a cache server.
  • the device includes:
  • a receiving module configured to receive a storage request sent by the first terminal, where the storage request carries data to be stored
  • a storage module configured to store the data to be stored in the first solid-state hard disk in the cache server, and to correspond the data identification of the data to be stored to its first storage path in the first solid-state hard disk The relationship is stored in the first storage record;
  • the dump module is used to periodically dump the data to be dumped in the first solid-state hard disk to the cloud server, and to correspond the data identifier of the data to be dumped to the second storage path in the cloud server The relationship is stored in the second storage record.
  • the dump module is also used to periodically monitor the space utilization rate of the first solid state drive; when the space utilization rate is greater than the specified utilization rate, from the first solid state drive Select the data to be transferred and store the data to be transferred to the cloud server; record the transfer time of the data to be transferred, the difference between the current time and the transfer time exceeds
  • the data to be dumped is deleted from the first solid state hard disk, and the correspondence between the data identifier of the data to be dumped in the first storage record and the first storage path is deleted.
  • the dump module is further configured to select, from the first solid-state hard disk, data whose storage time is the farthest from the current time as the data of the first solid-state hard disk as the storage time Data to be transferred; or,
  • the dumping module is further configured to select the data with the smallest thermal value from the first solid-state hard disk as the data to be dumped according to the thermal value of the data in the first solid-state hard disk, and the thermal value of the data is used to indicate The probability of the data being read.
  • the device further includes:
  • a backup module configured to back up the data to be stored to the second solid-state hard disk in the backup server, and store the correspondence between the data identification of the data to be stored and the third storage path in the second solid-state hard disk In the third storage record.
  • the device further includes:
  • a deletion module configured to delete the data to be transferred from the second solid-state drive when the difference between the current time and the transfer time of the data to be transferred exceeds a second specified duration; The corresponding relationship between the data identifier of the data to be dumped and the third storage path in the third storage record is deleted.
  • the device further includes:
  • the receiving module is also used to receive a read request sent by the second terminal, where the read request carries a data identifier of the data to be read;
  • a reading module configured to read the data to be read according to the data identifier of the data to be read
  • the return module is configured to return the data to be read to the second terminal.
  • the reading module is further configured to determine whether the data identifier of the data to be read is included in the first storage record; when the first storage record includes the data identifier When reading the data identifier of the data, determine the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read, and according to the first of the data to be read A storage path to read the data to be read from the first solid-state hard disk; when the first storage record does not include the data identifier of the data to be read, according to the data of the data to be read Mark, determine the second storage path of the data to be read from the second storage record, and read the data to be read from the cloud server according to the second storage path of the data to be read .
  • the reading module is further configured to determine whether a data identifier of the data to be read is included in a third storage record, and the third storage record is a backup server of the cache server Storage record; when the third storage record contains the data identifier of the data to be read, determine the data to be read from the third storage record according to the data identifier of the data to be read.
  • the third storage path according to the third storage path of the data to be read, read the data to be read from the second SSD in the backup server; when the third storage record does not contain
  • the second storage path of the data to be read is determined from the second storage record according to the data identification of the data to be read.
  • an embodiment of the present application provides a data storage device based on cloud storage, the device includes
  • At least one processor At least one processor
  • At least one memory At least one memory
  • the at least one memory stores one or more programs, the one or more programs are configured to be executed by the at least one processor, and the one or more programs are included in any possible implementation manner described above
  • the instructions of the data storage method based on cloud storage.
  • an embodiment of the present application provides a non-volatile computer-readable storage medium for storing a computer program, the computer program is loaded by a processor to execute as described in any possible implementation manner described above Instructions for data storage methods based on cloud storage.
  • a cache server is provided in the cloud storage system, and a solid-state drive is used to store data in the cache server, and the characteristics of the solid-state drive can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
  • FIG. 1 is a schematic structural diagram of a cloud storage system provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of another cloud storage system provided by an embodiment of the present application.
  • FIG. 3 is a method flowchart of a data storage method based on cloud storage provided by an embodiment of the present application
  • FIG. 4 is a method flowchart of a data storage method based on cloud storage provided by an embodiment of the present application
  • FIG. 5 is a method flowchart of a cloud storage-based data storage method provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a cloud storage data storage device provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a cache server provided by an embodiment of the present application.
  • FIG. 1 is a cloud storage system provided by an embodiment of the present application.
  • the cloud storage system includes: a first terminal 101, a second terminal 102, a cache server 103, and a cloud server 104.
  • the first terminal 101 and the second terminal 102 are respectively connected to the cache server 103 through a network, and the cache server 103 and the cloud server 104 are connected through a network.
  • the first terminal 101 is a terminal that stores data.
  • the second terminal 102 is a terminal that reads data.
  • the first terminal 101 and the second terminal 102 may be the same terminal or different terminals.
  • the cloud server 104 may be a cloud storage system.
  • the first terminal 101 is used to upload data to be stored to the cache server 103.
  • the cache server 103 is used to store the data to be stored in the first solid-state hard disk included in it, and store the correspondence between the data identification of the data to be stored and its first storage path in the first solid-state hard disk in the first storage record in.
  • the cache server 103 is also used to periodically transfer the data to be transferred from the first solid-state drive to the cloud server 104, and store the correspondence between the data identifier of the data to be transferred and the second storage path in the cloud server 104 in the first Two storage records.
  • the cache server 103 transfers the data to be transferred from the first solid-state drive to the cloud server 104, and the data to be transferred may be storage Old data with a long period of time or data with relatively low heat.
  • the cache server 103 deletes the data to be transferred from the first solid-state drive after a period of time after storing the data to be transferred into the first solid-state drive.
  • the cloud server 104 since the cloud server 104 generally uses an ordinary mechanical hard disk, after the cache server 103 transfers the data to be dumped to the cloud server 104, the data to be stored in the first solid-state hard disk cannot be deleted immediately, to avoid the data of the mechanical hard disk of the cloud server 104 Failure to implement refresh orders will take effect.
  • the cache server 103 deletes the data to be transferred from the first solid-state hard disk, and releases the storage space in the first solid-state hard disk, so as to continuously cache the newly uploaded data of the first terminal 101.
  • the second terminal 102 is used to download the data to be read to the cache server 103.
  • the cache server 103 determines the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read, according to the The first storage path of the data to be read reads the data to be read from the first solid-state hard disk.
  • the cache server 103 determines the second storage path of the data to be read from the second storage record according to the data identifier according to the data to be read, and according to the data to be read The second storage path, read the data to be read from the cloud server 104.
  • a cache server 103 is provided in the cloud storage system, and a solid-state drive is used to store data in the cache server 103, and the characteristics of the solid-state drive can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal 101 stores data, the cache server 103 preferentially writes the data to be stored into the cache server 103, and then the cache server 103 transfers the data asynchronously to the cloud server 104 for persistent storage. Therefore, the cache server 103 can quickly store and download data, and thus can support the user's demand for data to be stored and retrieved.
  • the cloud storage system further includes a backup server 105, and the backup server 105 and the cache server 103 are connected through a network.
  • the cache server 103 stores the data to be stored in the first solid-state hard disk included in it
  • the data to be stored is stored in the second solid-state hard disk included in the backup server 105, thereby further improving the security of the data to be stored.
  • the cache server 103 deletes the data to be stored from the first solid-state drive
  • the data to be stored is also deleted from the second solid-state drive, so that the second solid-state drive can continue to cache newly uploaded data from the first terminal 101 data.
  • An embodiment of the present application provides a data storage method based on cloud storage, and the method is applied to a cache server.
  • the method includes:
  • Step 301 Receive a storage request sent by a first terminal, where the storage request carries data to be stored.
  • Step 302 Store the data to be stored in the first solid state drive in the cache server, and store the correspondence between the data identifier of the data to be stored and its first storage path in the first solid state drive In the first storage record.
  • Step 303 Periodically transfer the data to be dumped in the first solid-state hard disk to the cloud server, and store the correspondence between the data identifier of the data to be dumped and its second storage path in the cloud server in In the second storage record.
  • the periodically dumping the data to be dumped in the first solid state disk to the cloud server includes:
  • the selecting the data to be dumped from the first solid state drive includes:
  • the storage time of the data in the first solid state drive select the data with the furthest storage time away from the current time as the data to be transferred from the first solid state drive; or,
  • the data with the smallest thermal value is selected from the first solid state drive as the data to be dumped, and the thermal value of the data is used to indicate the probability of the data being read.
  • the method further include:
  • the method further includes :
  • the method further includes:
  • the reading the data to be read according to the data identifier of the data to be read includes:
  • the first storage of the data to be read is determined from the first storage record according to the data identifier of the data to be read Path, according to the first storage path of the data to be read, reading the data to be read from the first solid state drive;
  • the second identifier of the data to be read is determined from the second storage record according to the data identifier of the data to be read
  • the storage path reads the data to be read from the cloud server according to the second storage path of the data to be read.
  • the method before determining the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, the method further includes:
  • the third storage record contains the data identifier of the data to be read, and the third storage record is a storage record of the backup server of the cache server;
  • the third storage of the data to be read is determined from the third storage record according to the data identification of the data to be read Path, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server;
  • a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
  • An embodiment of the present application provides a data storage method based on cloud storage.
  • a first terminal is used to store data as an example for illustration.
  • the method is applied between a first terminal, a cache server, a cloud server, and a backup server. .
  • the method includes:
  • Step 401 The first terminal sends a storage request to the cache server, where the storage request carries data to be stored.
  • the first terminal runs a cloud storage application, logs in to the cache server based on the cloud storage application, and sends a storage request to the cache server, where the storage request carries data to be stored.
  • the data to be stored may be data in any format.
  • the data to be stored may be text, audio data, video data, pictures, and so on.
  • the data to be stored may be data already stored by the first terminal, or may be data currently received by the first terminal; the data to be stored may also be a chat record of a social application, or data in a specified format in the social application ( Pictures or videos, etc.).
  • the data to be stored when the data to be stored is a picture, the data to be stored may be a picture that has been stored in the picture library, or a picture that is currently taken.
  • the data to be stored is a picture
  • the data to be stored may be a picture currently received through a social application.
  • the heat value of the data to be stored is carried in the storage request, but also the heat value of the data to be stored may be carried in the storage request.
  • the heat value of the data to be stored is used to indicate the probability of the data to be stored being read.
  • the first terminal may upload the data to be stored to the cache server when the user triggers the storage instruction, which is the first implementation manner.
  • the first terminal can also automatically upload the unuploaded data to be stored to the cache server, that is, the second implementation manner.
  • this step may be: the first terminal receives the storage instruction, displays an upload interface, and the upload interface includes a data index of multiple data.
  • the first terminal acquires the selected data index, acquires data corresponding to the data index based on the data index, and uses the acquired data as data to be stored.
  • the upload interface also includes an upload button; when the first terminal detects that the upload button is triggered, it sends a storage request to the cache server, where the storage request carries the data to be stored.
  • this step may be: the first terminal obtains unuploaded data to be stored from the database, and sends a storage request to the cache server, where the storage request carries the data to be stored.
  • the database may be a picture library.
  • the first terminal can automatically identify the unuploaded data to be stored, automatically upload the unuploaded data to the cache server, and then store it in the cloud server, and the production can be realized without the participation of the user. , Not only improve upload efficiency, but also increase user viscosity.
  • the first terminal needs to consume more network resources to send the storage request to the cache server.
  • the first terminal determines the network type that the first terminal is currently connected to; when the network type is a wireless network, the step of sending the storage request from the first terminal to the cache server is performed; when the network type When it is a mobile network, the first terminal temporarily does not send a storage request to the cache server, and only sends a storage request to the cache server when it detects a wireless network.
  • Step 402 The cache server receives the storage request of the first terminal, stores the data to be stored in the first solid state drive, and the correspondence between the data identifier of the data to be stored and its first storage path in the first solid state drive Stored in the first storage record.
  • the cache server includes a first solid state drive.
  • the cache server stores the data to be stored carried in the storage request in the first solid state drive, determines the first storage path of the data to be stored in the first solid state drive, and identifies the data of the data to be stored The corresponding relationship with the first storage path is stored in the first storage record.
  • the cache server may include one solid state drive or multiple solid state drives; when the cache server includes one solid state drive, the cache server may directly store the data to be stored in the one solid state drive; when the cache server includes multiple In the case of a solid-state drive, the cache server may select one solid-state drive from multiple solid-state drives, and store the data to be stored in the selected solid-state drive.
  • the cache server when the cache server selects the solid-state drive, the cache server can randomly select one solid-state drive from multiple solid-state drives; it can also select a solid-state drive with the lowest space utilization rate from the multiple solid-state drives; Priority level and space utilization of each solid state drive, select a solid state drive that matches the priority level from multiple solid state drives. For example, when the priority level of the first terminal is high, select a solid state drive with a low space utilization rate from multiple solid state drives; when the priority level of the first terminal is low, select a space utilization rate from multiple solid state drives Higher solid state drive.
  • the cache server can also select a solid-state drive that matches the thermal value from multiple solid-state drives according to the thermal value of the data to be stored and the space utilization of each solid-state drive; for example, when the thermal value of the data to be stored is high When selecting the SSD with high space utilization from multiple SSDs; when the heat value of the data to be stored is low, select the SSD with low space utilization from among multiple SSDs.
  • the cache server after the cache server stores the data to be stored in the first solid-state disk, the storage time of the data to be stored is obtained, and the data identifier and storage time of the data to be stored are associated to facilitate subsequent Data storage data transfers the data.
  • the cache server when the storage request also carries the heat value of the data to be stored, the cache server also stores the correspondence between the data identifier of the data to be stored and the heat value, so as to subsequently obtain the heat value of the data to be stored.
  • a backup server may be set for the cache server. Therefore, after step 402 is executed, step 403 is executed, and then step 404 is executed. In addition, when no backup server is provided for the cache server, after step 402 is executed, step 404 is directly executed.
  • the cache server When the cache server stores the data to be stored in the first solid-state hard disk, it will return a successful storage response to the first terminal, where the successful storage response is used to indicate that the data to be stored has been successfully stored.
  • the successful storage response is used to indicate that the data to be stored has been successfully stored.
  • the storage speed of the first SSD is extremely fast. Therefore, storing the data to be stored in the first solid-state hard disk returns a successful storage response to the first terminal. For the user, the response time is shortened and the user viscosity is increased.
  • Step 403 The cache server backs up the data to be stored to the second solid state drive in the backup server, and stores the correspondence between the data identifier of the data to be stored and its third storage path in the second solid state drive in the third Store records.
  • the cache server sends a backup request to the backup server, where the backup request carries the data to be stored.
  • the backup server receives the backup request sent by the cache server, stores the data to be stored in the second solid-state drive included therein, determines a third storage path of the data to be stored in the second solid-state drive, and stores the data of the data to be stored The correspondence between the identifier and the third storage path is stored in the third storage record.
  • the backup server may return the third storage record to the cache server.
  • the cache server receives the third storage record returned by the backup server.
  • the backup server may return the third storage path to the cache server.
  • the cache server stores the correspondence between the data identifier of the data to be stored and the third storage path in the third storage record, so as to facilitate subsequent data location query.
  • the cache server when the cache server stores the data to be stored in the first solid state drive, it can back up the data to be stored to the second solid state drive in the backup server; the cache server can also periodically store the first The data in the solid state drive is backed up to the second solid state drive in the backup server.
  • the cache server may back up all data in the first solid state drive to the backup server, or may only back up more important data to the backup server.
  • the cache server determines the importance of the data to be stored; only when the importance of the data to be stored exceeds the second preset threshold. The step of the cache server backing up the data to be stored to the first solid-state hard disk in the backup server is performed; when the important program of the data to be stored does not exceed the first preset threshold, the cache server does not perform the backup operation.
  • the step of the cache server determining the importance of the data to be stored may be: the cache server may use the heat value of the data to be stored as the importance of the data to be stored.
  • the cache server may also determine the importance of the data to be stored according to the content of the data to be stored; for example, when the content of the data to be stored is a bank bill, determine the importance of the data to be stored to a large value; When the content of the data to be stored is image information without persons, it is determined that the importance of the data to be stored is a small value.
  • Step 404 The cache server periodically transfers the data to be transferred from the first solid-state drive to the cloud server, and stores the correspondence between the data identifier of the data to be transferred and its second storage path in the cloud server in the second storage record in.
  • the cache server regularly monitors the space utilization rate of the first solid-state drive, and performs the dump processing only when the space utilization rate is high, so as to store the data in the first solid-state drive as much as possible, so that subsequent The data can be directly read from the first solid-state hard disk, which improves the reading efficiency.
  • this step can be achieved by the following steps (1) to (4), including:
  • the cache server regularly monitors the space utilization of the first solid state drive.
  • the cache server monitors the space utilization rate of the first solid-state drive every third specified time period to determine whether the space utilization rate is greater than the specified utilization rate; when the space utilization rate is greater than the specified utilization rate, step (2) is executed; when the space When the utilization rate is not greater than the specified utilization rate, continue to perform step (1) until the space utilization rate of the first solid state drive is greater than the specified utilization rate.
  • the third specified duration and specified utilization can be set and changed as needed.
  • the third specified duration and specified utilization are not specifically limited; for example, the third specified duration can be 12 hours or 1 Wait.
  • the specified utilization rate can be 85% or 90%.
  • the cache server selects the data to be transferred from the first solid state disk.
  • step (2) can be: the cache server determines the storage data in the first solid state drive, and according to the storage time of the data in the first solid state drive, from The first solid-state hard disk selects the data whose storage time is farthest from the current time, and uses the selected data as the data to be transferred. or,
  • the cache server determines the data stored in the first solid-state drive, and selects the data whose time difference between the storage time and the current time reaches the fifth specified duration from the first solid-state drive according to the storage time of the data in the first solid-state drive. Time is used as data to be transferred.
  • the cache server has stored the storage time of the data in the first solid state disk.
  • the step of the cache server determining the stored data of the data in the first solid state drive may be: the cache server determines the data in the first solid state drive from the correspondence between the data identification and the storage time according to the data identification of the data in the first solid state drive Storage time.
  • step (2) may be: the cache server determines the heat value of the data in the first solid state drive, and according to the heat value of the data in the first solid state drive, selects the data with the smallest heat value from the first solid state drive to be transferred data.
  • the heat value of the data is used to indicate the probability of the data being read. or,
  • the cache server determines the heat value of the data in the first solid state drive, and selects data with a heat value less than the second preset threshold from the first solid state drive as the data to be dumped according to the heat value of the data in the first solid state drive.
  • the cache server has stored the correspondence between the data identifier of the data and the heat value.
  • the step of the cache server determining the heat value of the data in the first solid state drive may be: according to the data identifier of the data to be dumped, the cache server obtains the heat of the data to be dumped from the corresponding relationship between the data identifier and the heat value value.
  • the cache server records the transfer time of the data to be transferred, and deletes the data to be transferred from the first solid state drive when the difference between the current time and the transfer time exceeds the first specified duration Delete the corresponding relationship between the data identifier of the data to be dumped in the first storage record and the first storage path.
  • the cache server records the correspondence between the data identifier in the first solid state drive and the first storage path; correspondingly, the cache server may delete the data to be transferred from the first solid state drive as follows:
  • the data identifier of the data, the first storage path of the data to be dumped is obtained from the correspondence between the data identifier and the first storage path; according to the first storage path of the data to be dumped, the storage space corresponding to the first storage path Delete the data to be transferred.
  • the cache server stores the data to be dumped in the cloud server, it regularly inspects the data that has been dumped in the first solid-state drive, and when the current time is longer than the transfer-out time by more than the first specified time, Only after deleting the data in the first solid-state hard disk, the storage space in the first solid-state hard disk is released, so as to continuously cache the newly uploaded data of the user.
  • the first designated duration can be set and changed as needed. In the embodiments of the present application, the first designated duration is not specifically limited; for example, the first designated duration can be 5 minutes or 10 minutes.
  • the cloud server usually uses ordinary mechanical hard disks to store data. Therefore, after storing the data to be transferred to the cloud server, the cache server does not immediately delete the data to be transferred in the first solid state drive, but delays deleting the data to be transferred in the first solid state drive, thereby avoiding the cloud server
  • the mechanical hard disk has not been refreshed and placed in real time to take effect.
  • the cache server stores the data to be dumped in the cloud server, and stores the correspondence between the data identifier of the data to be dumped and its second storage path in the cloud server in the second storage record.
  • the cache server sends the data to be transferred to the cloud server.
  • the cloud server receives the data to be transferred sent by the cache server, stores the data to be transferred, determines the second storage path of the data to be transferred in the cloud server, and identifies the data identifier and the second storage path of the data to be transferred The corresponding relationship is stored in the second storage record.
  • the cloud storage server may send the second storage record to the cache server.
  • the cloud server receives the second storage record and stores the second storage record.
  • the cloud server may return the second storage path to the cache server.
  • the cache server stores the correspondence between the data identifier of the data to be stored and the second storage path in the second storage record, so as to facilitate subsequent data location query.
  • the cache server may not perform the transfer process until the space utilization rate in the first solid-state drive reaches the specified utilization rate, but instead transfers the first solid-state drive to be transferred every fourth specified duration
  • the stored data is transferred to the cloud server.
  • the fourth designated duration can be set and changed as needed. In the embodiments of the present application, the fourth designated duration is not specifically limited; for example, the fourth designated duration can be 1 day or 2 days.
  • step 405 is executed to notify the backup server to delete the same data.
  • Step 405 When the difference between the current time and the transfer time of the data to be transferred exceeds the second specified duration, the cache server deletes the data to be transferred from the second solid-state hard disk and stores the third The corresponding relationship between the data identifier of the data to be transferred and the third storage path in the record is deleted.
  • the cache server sends the data identifier of the data to be dumped to the backup server to notify the backup server to delete the data to be dumped from the second solid-state hard disk.
  • the step of the cache server deleting the data to be dumped from the second solid state disk may be: the cache server sends a first deletion request to the backup server, and the first deletion request carries the data identifier of the dumped data.
  • the backup server receives the first deletion request of the cache server, obtains the third storage path of the data to be transferred from the correspondence between the data identification and the third storage path according to the data identification of the data to be transferred, and stores the third storage path according to the data to be transferred
  • the third storage path of the data deletes the data to be transferred from the storage space corresponding to the third storage path.
  • the corresponding relationship between the data identifier and the third storage path is stored in the cache server, and the cache server may obtain the third storage path of the data to be dumped in the second solid-state drive and directly notify the backup server Delete the dumped data in the storage space corresponding to the third storage path.
  • the step of the cache server deleting the data to be dumped from the second solid state disk may be: the cache server sends a second deletion request to the backup server, and the second deletion request carries the data identifier of the dumped data.
  • the backup server receives the second deletion request of the cache server, and deletes the data to be transferred in the storage space corresponding to the third storage path according to the third storage path of the data to be transferred.
  • the cache server deletes the data to be transferred in the first solid-state drive
  • the data in the second solid-state drive is deleted and the second solid-state drive is released only when the current time exceeds the transfer-out time by a second specified time
  • the second specified duration can be set and changed as needed.
  • the second specified duration is not specifically limited; for example, the second specified duration can be 5 minutes or 10 minutes.
  • a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
  • An embodiment of the present application provides a data storage method based on cloud storage.
  • the second terminal reads data as an example for description.
  • the method is applied between the second terminal, cache server, cloud server and backup server. Referring to Figure 5, the method includes:
  • Step 501 The second terminal sends a read request to the cache server, where the read request carries the data identifier of the data to be read.
  • Run a cloud storage application in the second terminal log in to the cache server based on the cloud storage application, and when the second terminal downloads data from the cloud server, the second terminal sends a read request to the cache server, the read request carries a pending read The data identification of the data.
  • the first solid-state drive used in the cache server stores data
  • the first solid-state drive supports instant storage and retrieval
  • the cache server receives the read request, it first determines whether the data to be read is stored in the first solid-state drive ; If the data to be read is stored in the first solid state drive, the data to be read is read from the first solid state drive.
  • the cache server determines whether the data to be read is stored in the second solid state drive in the backup server. If the data to be read is stored in the second solid state drive, the data to be read is read from the second solid state drive. If the data to be read is not stored in the second solid-state hard disk, the data to be read is read from the cloud server.
  • the cache server determines to read the data to be read directly from the cloud server.
  • Step 502 The cache server receives the read request sent by the second terminal, and determines whether the data identifier of the data to be read is included in the first storage record.
  • the first storage record includes the correspondence between the data stored in the first solid-state hard disk in the cache server and the first storage path. Therefore, the cache server may determine whether the data to be read is stored in the first solid state drive by determining whether the data identifier of the data to be read is included in the first storage record.
  • step 503 When the data identifier of the data to be read is included in the first storage record, it is determined that the data to be read is stored in the first solid state drive, and step 503 is performed; when the first storage record does not contain the data to be read If there is a data identifier of the server and there is a backup server, go to step 504; when the first storage record does not contain the data identifier of the data to be read and there is no backup server, and there is no backup server, go to step 506.
  • Step 503 When the first storage record includes the data identifier of the data to be read, the cache server determines the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read , According to the first storage path of the data to be read, read the data to be read from the first solid state drive, and step 507 is executed.
  • Step 504 When the first storage record does not contain the data identification of the data to be read, the cache server determines whether the third storage record contains the data identification of the data to be read, and the third storage record is the cache server Storage records of the backup server.
  • the cache server directly determines whether the data identifier of the data to be read is included in the locally stored third storage record.
  • the step of the cache server determining whether the data identifier of the data to be read is included in the third storage record may be: the cache server sends a query request to the backup server, the query request carrying the query The data identification of the data.
  • the backup server receives the query request from the cache server, determines whether the data identifier of the data to be read is contained in the third storage record, and returns a query result to the cache server, which is the data containing the data to be read from the third storage record The identifier, or the third storage record does not contain the data identifier of the data to be read.
  • step 505 When the data identifier of the data to be read is included in the third storage record, step 505 is performed; when the data identifier of the data to be read is not included in the third storage record, step 506 is performed.
  • the backup server determines that the data identifier of the data to be read is contained in the third storage record, it can also directly read the data to be read according to the data identifier of the data to be read, and the query result Directly carry the data to be read, thereby further improving the reading efficiency.
  • Step 505 When the third storage record contains the data identifier of the data to be read, the cache server determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read , According to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server, and step 507 is executed.
  • the cache server sends the data identifier of the data to be read to the backup server to read the data to be read from the second solid-state hard disk in the backup server.
  • this step may be: the cache server sends a first download request to the backup server, and the first download request carries the data identifier of the data to be read.
  • the backup server receives the first download request of the cache server, determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read, and according to the third storage of the data to be read Path to read the data to be read from the second solid-state hard disk in the backup server.
  • the backup server returns the data to be read to the cache server.
  • the corresponding relationship between the data identifier and the third storage path is stored in the cache server, and the cache server may obtain the third storage path of the data to be dumped in the second solid-state drive and directly notify the backup server The dumped data from the storage space corresponding to the third storage path.
  • this step can be:
  • the cache server determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read, and sends a second download request to the backup server, where the second download request carries the data to be read Third storage path.
  • the backup server receives the second download request of the cache server, and reads the data to be read from the second solid-state hard disk in the backup server according to the third storage path of the data to be read.
  • the backup server returns the data to be read to the cache server.
  • Step 506 When the third storage record does not contain the data identifier of the data to be read, the cache server determines the second storage of the data to be read from the second storage record according to the data identifier of the data to be read Path, read the data to be read from the cloud server according to the second storage path of the data to be read, and perform step 507.
  • the cache server sends the data identifier of the data to be read to the cloud server to read the data to be read from the cloud server.
  • this step may be: the cache server sends a third download request to the cloud server, and the third download request carries the data identifier of the data to be read.
  • the cloud server receives the third download request of the cache server, determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and according to the second storage of the data to be read Path to read the data to be read from the cloud server.
  • the cloud server returns the data to be read to the cache server.
  • the corresponding relationship between the data identifier and the second storage path is stored in the cache server, and the cache server can obtain the second storage path of the data to be transferred in the cloud server, and directly notify the cloud server to The dumped data in the storage space corresponding to the second storage path.
  • this step can be:
  • the cache server determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and sends a fourth download request to the cloud server, where the fourth download request carries the data to be read Second storage path.
  • the cloud server receives the fourth download request of the cache server, and reads the data to be read from the cloud server according to the second storage path of the data to be read.
  • the cloud server returns the data to be read to the cache server.
  • step 506 when there is no backup server and it is determined in step 502 that the first storage record does not contain the data identifier of the data to be read, step 506 is performed, and step 506 can be replaced by: When a storage record does not contain the data identifier of the data to be read, the cache server determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and Read the second storage path of the data, read the data to be read from the cloud server, and perform step 507.
  • Step 507 The cache server returns the data to be read to the second terminal.
  • a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the second terminal reads the data, the cache server preferentially reads the data to be read from the cache server. When the cache server does not store the data to be read, the data to be read is read from the backup server. When the data to be read is not stored in the backup server, the cloud server is used to read the data to be read. Since both the cache server and the backup server use solid-state hard disks to store data, data can be quickly downloaded from the cache server or the backup server, thereby supporting the user's demand for data to be stored and retrieved.
  • An embodiment of the present application provides a cloud storage-based data storage device.
  • the device is applied to a cache server, and is used to perform the steps performed by the cache server in the cloud storage-based data storage method.
  • the device includes:
  • the receiving module 601 is configured to receive a storage request sent by a first terminal, where the storage request carries data to be stored;
  • the storage module 602 is configured to store the data to be stored in the first solid state drive in the cache server, and identify the data of the data to be stored and its first storage path in the first solid state drive The corresponding relationship is stored in the first storage record;
  • the dump module 603 is used to periodically dump the data to be dumped in the first solid-state hard disk to the cloud server, and to identify the data identifier of the data to be dumped and its second storage path in the cloud server The correspondence is stored in the second storage record.
  • the dump module 603 is further used to periodically monitor the space utilization rate of the first solid state drive; when the space utilization rate is greater than the specified utilization rate, the first solid state drive Select the data to be transferred, and store the data to be transferred to the cloud server; record the transfer time of the data to be transferred, the difference between the current time and the transfer time When the first specified time period is exceeded, the data to be dumped is deleted from the first solid state drive, and the correspondence between the data identifier of the data to be dumped in the first storage record and the first storage path is deleted .
  • the dump module 603 is further configured to select the data whose storage time is farthest from the current time from the first solid-state drive according to the storage time of the data in the first solid-state drive As data to be transferred; or,
  • the dump module 603 is further configured to select the data with the smallest thermal value from the first solid-state hard disk as the data to be dumped according to the thermal value of the data in the first solid-state hard disk, and the thermal value of the data is used to indicate The probability of the data being read.
  • the device further includes:
  • the backup module 604 is used for backing up the data to be stored to the second solid-state hard disk in the backup server, and mapping the data identification of the data to be stored to its third storage path in the second solid-state hard disk Stored in the third storage record.
  • the device further includes:
  • the deletion module 605 is configured to delete the data to be transferred from the second solid-state drive when the difference between the current time and the transfer time of the data to be transferred exceeds a second specified duration; The corresponding relationship between the data identifier of the data to be dumped and the third storage path in the third storage record is deleted.
  • the device further includes:
  • the receiving module 601 is further configured to receive a read request sent by a second terminal, where the read request carries a data identifier of data to be read;
  • a reading module 606, configured to read the data to be read according to the data identifier of the data to be read;
  • the return module 607 is configured to return the data to be read to the second terminal.
  • the reading module 606 is further configured to determine whether the data identifier of the data to be read is included in the first storage record; when the first storage record includes the data identifier When the data identification of the data to be read is performed, the first storage path of the data to be read is determined from the first storage record according to the data identification of the data to be read, and according to the first A storage path to read the data to be read from the first solid-state hard disk; when the first storage record does not include the data identifier of the data to be read, according to the data of the data to be read Data identification, determining the second storage path of the data to be read from the second storage record, and reading the to-be-read from the cloud server according to the second storage path of the data to be read data.
  • the reading module 606 is further configured to determine whether a data identifier of the data to be read is included in a third storage record, and the third storage record is a backup of the cache server The storage record of the server; when the third storage record contains the data identifier of the data to be read, determine the to-be-read from the third storage record according to the data identifier of the data to be read The third storage path of the data, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server; when the third storage record does not When the data identification of the data to be read is included, the second storage path of the data to be read is determined from the second storage record according to the data identification of the data to be read.
  • a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
  • the cloud storage-based data storage device provided in the above embodiment is based on the cloud storage data storage, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • the cloud storage-based data storage device provided in the foregoing embodiment and the cloud storage-based data storage method embodiment belong to the same concept. For the specific implementation process, see the method embodiment, and details are not described here.
  • FIG. 10 is a schematic structural diagram of a cache server provided by an embodiment of the present application.
  • the cache server 1000 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units (CPU) 1001 And one or more memories 1002, wherein at least one instruction is stored in the memory 1002, and the at least one instruction is loaded and executed by the processor 1001 to implement the methods provided by the foregoing method embodiments.
  • the server may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output.
  • the server may also include other components for implementing device functions, which will not be repeated here.
  • An embodiment of the present application provides a non-volatile computer-readable storage medium for storing a computer program, and the computer program is loaded by a processor to execute the instructions of the cloud storage-based data storage method.
  • the program may be stored in a computer-readable storage medium.
  • the mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种基于云存储的数据存储方法、装置和存储介质,属于云存储技术领域。该方法包括:缓存服务器接收第一终端发送的存储请求,该存储请求携带待存储数据;将该待存储数据存储在该缓存服务器中的第一固态硬盘中,将该待存储数据的数据标识与其在该第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;定期将该第一固态硬盘中待转存数据转存至云服务器中,将该待转存数据的数据标识与其在该云服务器中的第二存储路径的对应关系存储在第二存储记录中。缓存服务器优先将待存储数据写入缓存服务器,再由缓存服务器后台异步转存至云服务器进行持久化存储,从而缓存服务器可以快速存储和下载数据,支持用户数据即存即取的需求。

Description

基于云存储的数据存储方法、装置和存储介质
本申请要求于2018年11月13日提交、申请号为201811348900.X、发明名称为“基于云存储的数据存储方法、装置和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本公开中。
技术领域
本申请涉及云存储技术领域,特别涉及一种基于云存储的数据存储方法、装置和存储介质。
背景技术
随着计算机和网络的不断发展及高清视频的普及,存储数据所需的空间不断的增长,传统的存储结构已经很难满足用户需求。云存储以其海量的存储空间、高可靠性、高并发性、高可扩展性及高性价比等特性逐渐崭露头角。因此,越来越多的用户选择将数据存储在云服务器中。
目前,云服务器中一般使用机械硬盘来存储数据,云服务器接收到待存储的数据时,将待存储的数据存储到机械硬盘中,然后将机械硬盘中的数据刷新到物理硬盘中。
由于机械硬盘自身特性无法保证数据实时刷新到物理硬盘中,导致云服务器无法支持用户数据即存即取需求。
发明内容
本申请提供了一种基于云存储的数据存储方法、装置和存储介质,可以解决云服务器无法支持用户数据即存即取需求的问题。技术方案如下:
一方面,本申请实施例提供了一种基于云存储的数据存储方法,所述方法应用于缓存服务器,所述方法包括:
接收第一终端发送的存储请求,所述存储请求携带待存储数据;
将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存 储在第一存储记录中;
定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,所述定期将所述第一固态硬盘中待转存数据转存至云服务器中,包括:
定期监测所述第一固态硬盘的空间利用率;
当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;
记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
在另一个可能的实现方式中,所述从所述第一固态硬盘中选择所述待转存数据,包括:
根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
在另一个可能的实现方式中,所述将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中之后,所述方法还包括:
将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
在另一个可能的实现方式中,所述将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中之后,所述方法还包括:
在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;
将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对 应关系删除。
在另一个可能的实现方式中,所述方法还包括:
接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
根据所述待读取数据的数据标识,读取所述待读取数据;
向所述第二终端返回所述待读取数据。
在另一个可能的实现方式中,所述根据所述待读取数据的数据标识,读取所述待读取数据,包括:
确定所述第一存储记录中是否包含所述待读取数据的数据标识;
当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;
当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
在另一个可能的实现方式中,所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径之前,所述方法还包括:
确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;
当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;
当所述第三存储记录中不包含所述待读取数据的数据标识时,执行所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径的步骤。
另一方面,本申请实施例提供了一种基于云存储的数据存储装置,所述装置应用于缓存服务器,所述装置包括:
接收模块,用于接收第一终端发送的存储请求,所述存储请求携带待存储 数据;
存储模块,用于将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;
转存模块,用于定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,所述转存模块,还用于定期监测所述第一固态硬盘的空间利用率;当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
在另一个可能的实现方式中,所述转存模块,还用于根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
所述转存模块,还用于根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
在另一个可能的实现方式中,所述装置还包括:
备份模块,用于将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
在另一个可能的实现方式中,所述装置还包括:
删除模块,用于在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
在另一个可能的实现方式中,所述装置还包括:
所述接收模块,还用于接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
读取模块,用于根据所述待读取数据的数据标识,读取所述待读取数据;
返回模块,用于向所述第二终端返回所述待读取数据。
在另一个可能的实现方式中,所述读取模块,还用于确定所述第一存储记录中是否包含所述待读取数据的数据标识;当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
在另一个可能的实现方式中,所述读取模块,还用于确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;当所述第三存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径。
另一方面,本申请实施例提供了一种基于云存储的数据存储装置,所述装置包括
至少一个处理器;和
至少一个存储器;
所述至少一个存储器存储有一个或多个程序,所述一个或多个程序被配置成由所述至少一个处理器执行,所述一个或多个程序包含用于进行上述任一可能实现方式中所述的基于云存储的数据存储方法的指令。
另一方面,本申请实施例提供了一种非易失性计算机可读存储介质,用于存储计算机程序,所述计算机程序通过处理器进行加载来执行如上述任一可能实现方式中所述的基于云存储的数据存储方法的指令。
在本申请实施例中,在云存储系统中设置缓存服务器,且缓存服务器中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到 新的磁盘中。因此,第一终端在存储数据时,缓存服务器优先将待存储数据写入缓存服务器,再由缓存服务器后台异步转存至云服务器进行持久化存储。从而缓存服务器可以快速存储和下载数据,从而可以支持用户数据即存即取的需求。
附图说明
图1是本申请实施例提供的一种云存储系统的架构示意图;
图2是本申请实施例提供的另一种云存储系统的架构示意图;
图3是本申请实施例提供的一种基于云存储的数据存储方法的方法流程图;
图4是本申请实施例提供的一种基于云存储的数据存储方法的方法流程图;
图5是本申请实施例提供的一种基于云存储的数据存储方法的方法流程图;
图6是本申请实施例提供的一种云存储的数据存储装置的结构示意图;
图7是本申请实施例提供的另一种云存储的数据存储装置的结构示意图;
图8是本申请实施例提供的另一种云存储的数据存储装置的结构示意图;
图9是本申请实施例提供的另一种云存储的数据存储装置的结构示意图;
图10是本申请实施例提供的一种缓存服务器的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
图1是本申请实施例提供的一种云存储系统,参见图1,该云存储系统包括:第一终端101、第二终端102、缓存服务器103和云服务器104。第一终端101和第二终端102分别与缓存服务器103之间通过网络连接,缓存服务器103与云服务器104之间通过网络连接。第一终端101为存储数据的终端。第二终端102为读取数据的终端。第一终端101和第二终端102可以为相同终端,也可以为不同终端。本申请的一种实施方式中,云服务器104可以是一个云存储系统。
第一终端101用于向缓存服务器103上传待存储数据。缓存服务器103用于将该待存储数据存储在其包括的第一固态硬盘中,将该待存储数据的数据标识与其在第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中。缓存服务器103还用于定期将第一固态硬盘中待转存数据转存至云服务器104 中,将待转存数据的数据标识与其在云服务器104中的第二存储路径的对应关系存储在第二存储记录中。
其中,缓存服务器103的第一固态硬盘中的空间利用率达到指定利用率时,缓存服务器103将第一固态硬盘中的待转存数据转存至云服务器104中,待转存数据可以为存储时间较长的旧数据或者热度比较低的数据。缓存服务器103在将该待转存数据存储到第一固态硬盘中之后一段时间,才在第一固态硬盘中删除该待转存数据。由于云服务器104通常使用普通的机械硬盘,缓存服务器103将待转存数据转存到云服务器104之后,还不能立即删除第一固态硬盘中的待存储数据,避免云服务器104的机械硬盘数据还未实施刷新落盘生效。另外,缓存服务器103将第一固态硬盘中的待转存数据删除,释放第一固态硬盘中的存储空间,便于持续缓存第一终端101新上传的数据。
第二终端102用于向缓存服务器103下载待读取数据。当该待读取数据在缓存服务器103的第一固态硬盘中时,缓存服务器103根据待读取数据的数据标识,从第一存储记录中确定该待读取数据的第一存储路径,根据该待读取数据的第一存储路径,从第一固态硬盘中读取该待读取数据。当该待读取数据在云服务器104中时,缓存服务器103根据根据待读取数据的数据标识,从第二存储记录中确定该待读取数据的第二存储路径,根据该待读取数据的第二存储路径,从云服务器104中读取该待读取数据。
在本申请实施例中,在云存储系统中设置缓存服务器103,且缓存服务器103中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到新的磁盘中。因此,第一终端101在存储数据时,缓存服务器103优先将待存储数据写入缓存服务器103,再由缓存服务器103后台异步转存至云服务器104进行持久化存储。从而缓存服务器103可以快速存储和下载数据,从而可以支持用户数据即存即取的需求。
为了提升数据安全性,参见图2,该云存储系统中还包括备份服务器105,备份服务器105与缓存服务器103通过网络连接。缓存服务器103将待存储数据存储在其包括的第一固态硬盘中之后,将该待存储数据存储在备份服务器105包括的第二固态硬盘中,从而进一步提高待存储数据的安全性。
进一步的,缓存服务器103将待存储数据从第一固态硬盘中删除之后,将该待存储数据从第二固态硬盘中也删除,从而便于第二固态硬盘中可以持续缓 存第一终端101新上传的数据。
本申请实施例提供了一种基于云存储的数据存储方法,该方法应用在缓存服务器中。参见图3,该方法包括:
步骤301:接收第一终端发送的存储请求,所述存储请求携带待存储数据。
步骤302:将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中。
步骤303:定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,所述定期将所述第一固态硬盘中待转存数据转存至云服务器中,包括:
定期监测所述第一固态硬盘的空间利用率;
当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;
记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
在另一个可能的实现方式中,所述从所述第一固态硬盘中选择所述待转存数据,包括:
根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
在另一个可能的实现方式中,所述将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中之后,所述方法还包括:
将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在 第三存储记录中。
在另一个可能的实现方式中,所述将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中之后,所述方法还包括:
在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;
将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
在另一个可能的实现方式中,所述方法还包括:
接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
根据所述待读取数据的数据标识,读取所述待读取数据;
向所述第二终端返回所述待读取数据。
在另一个可能的实现方式中,所述根据所述待读取数据的数据标识,读取所述待读取数据,包括:
确定所述第一存储记录中是否包含所述待读取数据的数据标识;
当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;
当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
在另一个可能的实现方式中,所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径之前,所述方法还包括:
确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;
当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;
当所述第三存储记录中不包含所述待读取数据的数据标识时,执行所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径的步骤。
在本申请实施例中,在云存储系统中设置缓存服务器,且缓存服务器中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到新的磁盘中。因此,第一终端在存储数据时,缓存服务器优先将待存储数据写入缓存服务器,再由缓存服务器后台异步转存至云服务器进行持久化存储。从而缓存服务器可以快速存储和下载数据,从而可以支持用户数据即存即取的需求。
本申请实施例提供了一种基于云存储的数据存储方法,在该实施例中以第一终端存储数据为例进行说明,该方法应用在第一终端、缓存服务器、云服务器和备份服务器之间。参见图4,该方法包括:
步骤401:第一终端向缓存服务器发送存储请求,该存储请求携带待存储数据。
第一终端中运行云存储应用,基于该云存储应用登录该缓存服务器,向该缓存服务器发送存储请求,该存储请求携带待存储数据。该待存储数据可以为任一格式的数据。例如,该待存储数据可以为文本、音频数据、视频数据、图片等。且该待存储数据可以为第一终端已存储的数据,也可以为第一终端当前接收到的数据;该待存储数据还可以为社交应用的聊天记录,或者社交应用中的指定格式的数据(图片或者视频等)。例如,当该待存储数据为图片时,该待存储数据可以为图片库中已存储图片,也可以为当前拍摄得到的图片。再如,当该待存储数据为图片时,该待存储数据可以为通过社交应用当前接收到的图片。
另外,该存储请求中不仅携带待存储数据,该存储请求中还可以携带该待存储数据的热度值。该待存储数据的热度值用于指示该待存储数据被读取的概率大小。
第一终端可以在用户触发存储指令时,将待存储数据上传至缓存服务器,也即第一种实现方式。第一终端也可以实现自动将未上传的待存储数据上传至缓存服务器,也即第二种实现方式。
第一种实现方式,本步骤可以为:第一终端接收存储指令,显示上传界面, 该上传界面中包括多个数据的数据索引。第一终端获取被选择的数据索引,基于该数据索引获取该数据索引对应的数据,将获取的数据作为待存储的数据。该上传界面中还包括上传按钮;第一终端检测到该上传按钮被触发时,向缓存服务器发送存储请求,该存储请求携带待存储数据。
第二种实现方式,本步骤可以为:第一终端从数据库中获取未上传的待存储数据,向缓存服务器发送存储请求,该存储请求携带待存储数据。例如,当代存储数据为图片时,数据库可以为图片库。
在本申请实施例中,第一终端能够自动识别未上传的待存储数据,将未上传的待存储数据自动上传至缓存服务器,进而存储到云服务器中,不想要用户的参与即可实现上产,不仅提高了上传效率,还提高了用户粘度。
需要说明的是,由于该存储请求携带待存储数据。因此,第一终端向缓存服务器中发送存储请求需要消耗较多的网络资源。第一终端在向缓存服务器发送存储请求之前,确定该第一终端当前连接的网络类型;当该网络类型为无线网络时,才执行第一终端向缓存服务器发送存储请求的步骤;当该网络类型为移动网络时,第一终端暂时不向缓存服务器发送存储请求,当第一终端检测到无线网络时,才向缓存服务器中发送存储请求。
步骤402:缓存服务器接收第一终端的存储请求,将该待存储数据存储在第一固态硬盘中,将该待存储数据的数据标识和其在第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中。
缓存服务器中包括第一固态硬盘。缓存服务器接收到存储请求时,将该存储请求携带的待存储数据存储在第一固态硬盘中,确定该待存储数据在第一固态硬盘中的第一存储路径,将该待存储数据的数据标识和第一存储路径的对应关系存储在第一存储记录中。
其中,缓存服务器中可以包括一个固态硬盘或者多个固态硬盘;当缓存服务器中包括一个固态硬盘时,缓存服务器可以直接将该待存储数据存储到该一个固态硬盘中;当缓存服务器中包括多个固态硬盘时,缓存服务器可以从多个固态硬盘中选择一个固态硬盘,将该待存储数据存储到选择的固态硬盘中。
其中,缓存服务器在选择固态硬盘时,缓存服务器可以从多个固态硬盘中随机选择一个固态硬盘;还可以从多个固态硬盘中选择一个空间利用率最低的固态硬盘;还可以根据第一终端的优先级别和每个固态硬盘的空间利用率,从多个固态硬盘中选择与该优先级别匹配的固态硬盘。例如,当第一终端的优先 级别较高时,从多个固态硬盘中选择空间利用率较低的固态硬盘;当第一终端的优先级别较低时,从多个固态硬盘中选择空间利用率较高的固态硬盘。缓存服务器还可以根据该待存储数据的热度值和每个固态硬盘的空间利用率,从多个固态硬盘中选择与该热度值匹配的固态硬盘;例如,当该待存储数据的热度值较高时,从多个固态硬盘中选择空间利用率较高的固态硬盘;当待存储数据的热度值较低时,从多个固态硬盘中选择空间利用率较低的固态硬盘。
在一个可能的实现方式中,缓存服务器将该待存储数据存储至第一固态硬盘中之后,获取该待存储数据的存储时间,关联存储该待存储数据的数据标识和存储时间,以便于后续基于数据的存储数据对数据进行转存处理。另外,当该存储请求中还携带该待存储数据的热度值时,缓存服务器还存储该待存储数据的数据标识和热度值的对应关系,以便于后续获取该待存储数据的热度值。
在本申请实施例中,可以为缓存服务器设置备份服务器。因此,执行完步骤402之后,执行步骤403,然后执行步骤404。另外,当没有为缓存服务器设置备份服务器时,执行完步骤402之后,直接执行步骤404。
缓存服务器将该待存储数据存储在第一固态硬盘中时,就会向第一终端返回成功存储响应,该成功存储响应用于指示已经将该待存储数据已经存储成功。在本申请实施例中,不需要等待将该待存储数据存储在云服务器中就指示该待存储数据已经存储成功。由于第一固态硬盘的存储速度极快。因此,将待存储数据存储到第一固态硬盘中就向第一终端返回成功存储响应,对于用户而言,缩短了响应时间,提高了用户粘度。
步骤403:缓存服务器将该待存储数据备份至备份服务器中的第二固态硬盘中,将该待存储数据的数据标识与其在该第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
缓存服务器向备份服务器发送备份请求,该备份请求携带该待存储数据。备份服务器接收缓存服务器发送的备份请求,将该待存储数据存储在其包括的第二固态硬盘中,确定该待存储数据在第二固态硬盘中的第三存储路径,将该待存储数据的数据标识和第三存储路径的对应关系存储在第三存储记录中。
在一个可能的实现方式中,备份服务器将该待存储数据的数据标识和第三存储路径的对应关系存储在第三存储记录中之后,可以向缓存服务器返回第三存储记录。缓存服务器接收备份服务器返回的第三存储记录。另外,备份服务器确定出该待存储数据在第二固态硬盘中的第三存储路径之后,可以将该第三 存储路径返回给缓存服务器。缓存服务器将该待存储数据的数据标识和第三存储路径的对应关系存储在第三存储记录中,以便于后续进行数据位置查询。
需要说明的一点是,缓存服务器将该待存储数据存储在第一固态硬盘中时,就可以将该待存储数据备份至备份服务器中的第二固态硬盘中;缓存服务器也可以周期性将第一固态硬盘中的数据备份至备份服务器中的第二固态硬盘中。
需要说明的另一点是,在本步骤中,缓存服务器可以将第一固态硬盘中的数据均备份至备份服务器中,也可以仅备份比较重要的数据至备份服务器。相应的,缓存服务器将该待存储数据备份至备份服务器中的第二固态硬盘之前,缓存服务器确定该待存储数据的重要程度;当该待存储数据的重要程度超过第二预设阈值时,才执行缓存服务器将该待存储数据备份至备份服务器中的第一固态硬盘的步骤;当该待存储数据的重要程序没有超过第一预设阈值时,缓存服务器不执行备份操作。
其中,缓存服务器确定该待存储数据的重要程度的步骤可以为:缓存服务器可以将该待存储数据的热度值作为该待存储数据的重要程度。缓存服务器还可以根据该待存储数据的内容,确定该待存储数据的重要程度;例如,当该待存储数据的内容为银行账单时,确定该待存储数据的重要程度为一个较大的数值;当该待存储数据的内容为没有人物的图像信息时,确定该待存储数据的重要程度为一个较小的数值。
步骤404:缓存服务器定期将第一固态硬盘中待转存数据转存至云服务器中,将待转存数据的数据标识与其在云服务器中的第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,缓存服务器定期监测第一固态硬盘的空间利用率,在空间利用率较高时,才进行转存处理,从而尽可能将数据存储在第一固态硬盘中,这样后续可以直接从第一固态硬盘中读取数据,提高了读取效率。相应的,本步骤可以通过以下步骤(1)至(4)实现,包括:
(1):缓存服务器定期监测第一固态硬盘的空间利用率。
缓存服务器每隔第三指定时长监测一下第一固态硬盘的空间利用率,确定该空间利用率是否大于指定利用率;当该空间利用率大于指定利用率时,执行步骤(2);当该空间利用率不大于指定利用率时,继续执行步骤(1),直到第一固态硬盘的空间利用率大于指定利用率。
其中,第三指定时长和指定利用率可以根据需要进行设置并更改,在本申 请实施例中,对第三指定时长和指定利用率不作具体限定;例如,第三指定时长可以为12小时或者1天等。指定利用率可以为85%或者90%。
(2):当该空间利用率大于指定利用率时,缓存服务器从该第一固态硬盘中选择该待转存数据。
第一种实现方式,缓存服务器可以选择旧数据进行转存,相应的,步骤(2)可以为:缓存服务器确定第一固态硬盘中的存储数据,根据第一固态硬盘中数据的存储时间,从第一固态硬盘中选择存储时间离当前时间最远的数据,将选择的数据作为待转存数据。或者,
缓存服务器确定第一固态硬盘中的存储数据,根据第一固态硬盘中数据的存储时间,从第一固态硬盘中选择存储时间与当前时间之间的时间差达到第五指定时长的数据,将选择的时间作为待转存数据。
在步骤402中缓存服务器已经存储了第一固态硬盘中的数据的存储时间。相应的,缓存服务器确定第一固态硬盘中数据的存储数据的步骤可以为:缓存服务器根据第一固态硬盘中数据的数据标识,从数据标识和存储时间的对应关系中确定第一固态硬盘中数据的存储时间。
第二种实现方式,由于第一固态硬盘支持即存即取,因此,对于可能即存即取的数据先保留在第一固态硬盘中,而将不太可能即存即取的数据转存到云服务器中。相应的,步骤(2)可以为:缓存服务器确定第一固态硬盘中数据的热度值,根据第一固态硬盘中数据的热度值,从第一固态硬盘中选择热度值最小的数据作为待转存数据。其中,数据的热度值用于指示该数据被读取的概率大小。或者,
缓存服务器确定第一固态硬盘中数据的热度值,根据第一固态硬盘中数据的热度值,从第一固态硬盘中选择热度值小于第二预设阈值的数据作为待转存数据。
在步骤402中缓存服务器已经存储了数据的数据标识和热度值的对应关系。相应的,缓存服务器确定第一固态硬盘中数据的热度值的步骤可以为:缓存服务器根据该待转存数据的数据标识,从数据标识和热度值的对应关系中获取该待转存数据的热度值。
(3):缓存服务器记录该待转存数据的转存时间,在当前时间与该转存时间之间的差值超过第一指定时长时,将该待转存数据从第一固态硬盘中删除,将第一存储记录中该待转存数据的数据标识与第一存储路径的对应关系删除。
缓存服务器记录了第一固态硬盘中的数据标识和第一存储路径的对应关系;相应的,缓存服务器将该待转存数据从第一固态硬盘中删除的步骤可以为:缓存服务器根据待转存数据的数据标识,从数据标识和第一存储路径的对应关系中获取该待转存数据的第一存储路径;根据该待转存数据的第一存储路径,从第一存储路径对应的存储空间中删除该待转存数据。
需要说明的一点是,缓存服务器将该待转存数据存储到该云服务器中之后,定期巡检第一固态硬盘中已经转存的数据,在当前时间比转出时间超过第一指定时长时,才删除第一固态硬盘中的该数据,释放第一固态硬盘中的存储空间,便于持续缓存用户新上传的数据。其中,第一指定时长可以根据需要进行设置并更改,在本申请实施例中,对第一指定时长不作具体限定;例如,第一指定时长可以为5分钟或者10分钟。
需要说明的另一点是,由于云服务器中通常使用普通的机械硬盘存储数据。因此,缓存服务器将待转存数据存储到云服务器之后,并不立即删除第一固态硬盘中的待转存数据,而是延迟删除第一固态硬盘中的待转存数据,从而避免云服务器中的机械硬盘还未实时刷新落盘生效。
(4):缓存服务器将该待转存数据存储到该云服务器中,将待转存数据的数据标识与其在云服务器中的第二存储路径的对应关系存储在第二存储记录中。
缓存服务器向云服务器发送该待转存数据。云服务器接收缓存服务器发送的该待转存数据,存储该待转存数据,确定该待转存数据在云服务器中的第二存储路径,将该待转存数据的数据标识和第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,云服务器将待转存数据的数据标识和第二存储路径的对应关系存储在第二存储记录中之后,可以将第二存储记录发送给缓存服务器。云服务器接收第二存储记录,存储第二存储记录。另外,云服务器确定出该待存储数据在云服务器中的第二存储路径之后,可以将第二存储路径返回给缓存服务器。缓存服务器将该待存储数据的数据标识和第二存储路径的对应关系存储在第二存储记录中,以便于后续进行数据位置查询。
在另一个可能的实现方式中,缓存服务器可以不在第一固态硬盘中的空间利用率达到指定利用率时,才进行转存处理,而是每隔第四指定时长将第一固态硬盘中待转存数据转存至云服务器中。其中,第四指定时长可以根据需要进行设置并更改,在本申请实施例中,对第四指定时长不作具体限定;例如,第 四指定时长可以为1天或者2天等。
需要说明的一点是,当为缓存服务器设置备份服务器时,执行完步骤404之后,执行步骤405通知备份服务器删除同样的数据。当没有为缓存服务器设置备份服务器时,执行完步骤404之后,结束。
步骤405:在当前时间与该待转存数据的转存时间之间的差值超过第二指定时长时,缓存服务器将该待转存数据从该第二固态硬盘中删除,将该第三存储记录中的该待转存数据的数据标识与第三存储路径的对应关系删除。
在一个可能的实现方式中,缓存服务器向备份服务器发送该待转存数据的数据标识,以通知备份服务器将待转存数据从该第二固态硬盘中删除。相应的,缓存服务器将该待转存数据从该第二固态硬盘中删除的步骤可以为:缓存服务器向备份服务器发送第一删除请求,第一删除请求携带该转存数据的数据标识。备份服务器接收缓存服务器的第一删除请求,根据该待转存数据的数据标识,从数据标识和第三存储路径的对应关系中获取该待转存数据的第三存储路径,根据该待转存数据的第三存储路径,将第三存储路径对应的存储空间中的该待转存数据删除。
在另一个可能的实现方式中,缓存服务器中存储数据标识和第三存储路径的对应关系,缓存服务器可以获取到该待转存数据在第二固态硬盘中的第三存储路径,直接通知备份服务器删除第三存储路径对应的存储空间中的该转存数据。相应的,缓存服务器将该待转存数据从该第二固态硬盘中删除的步骤可以为:缓存服务器向备份服务器发送第二删除请求,第二删除请求携带该转存数据的数据标识。备份服务器接收缓存服务器的第二删除请求,根据该待转存数据的第三存储路径,将第三存储路径对应的存储空间中的该待转存数据删除。
需要说明的是,缓存服务器删除第一固态硬盘中的待转存数据之后,在当前时间比转出时间超过第二指定时长时,才删除第二固态硬盘中的该数据,释放第二固态硬盘中的存储空间,便于持续缓存用户新上传的数据。其中,第二指定时长可以根据需要进行设置并更改,在本申请实施例中,对第二指定时长不作具体限定;例如,第二指定时长可以为5分钟或者10分钟。
在本申请实施例中,在云存储系统中设置缓存服务器,且缓存服务器中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到新的磁盘中。因此,第一终端在存储数据时,缓存服务器优先将待存储数据写入缓存服务器,再由缓存服务器后台异步转存至云服务器进行持久化存储。从 而缓存服务器可以快速存储和下载数据,从而可以支持用户数据即存即取的需求。
本申请实施例提供了一种基于云存储的数据存储方法,在该实施例中以第二终端读取数据为例进行说明。该方法应用在第二终端、缓存服务器、云服务器和备份服务器之间。参见图5,该方法包括:
步骤501:第二终端向缓存服务器发送读取请求,该读取请求携带待读取数据的数据标识。
第二终端中运行云存储应用,基于该云存储应用登录该缓存服务器,在第二终端从云服务器中下载数据时,第二终端向缓存服务器发送读取请求,该读取请求携带待读取数据的数据标识。
由于缓存服务器中使用的第一固态硬盘来存储数据,而第一固态硬盘支持即存即取,因此,缓存服务器接收到读取请求时,优先确定第一固态硬盘中是否存储该待读取数据;如果第一固态硬盘中存储该待读取数据,则从第一固态硬盘中读取该待读取数据。
如果第一固态硬盘中不存储该待读取数据,且为缓存服务器设置了备份服务器时,缓存服务器确定备份服务器中的第二固态硬盘中是否存储该待读取数据。如果第二固态硬盘中存储该待读取数据,则从第二固态硬盘中读取该待读取数据。如果第二固态硬盘中也不存储该待读取数据时,才从云服务器中读取该待读取数据。
如果第一固态硬盘中不存储该待读取数据,且没有为缓存服务器设置备份服务器时,缓存服务器确定直接从云服务器中读取该待读取数据。
步骤502:缓存服务器接收第二终端发送的读取请求,确定第一存储记录中是否包含该待读取数据的数据标识。
第一存储记录中包括缓存服务器中的第一固态硬盘中存储的数据和第一存储路径的对应关系。因此,缓存服务器可以通过确定第一存储记录中是否包含该待读取数据的数据标识来确定第一固态硬盘中是否存储该待读取数据。当该第一存储记录中包含该待读取数据的数据标识时,确定第一固态硬盘中存储了该待读取数据,执行步骤503;当该第一存储记录中不包含该待读取数据的数据标识,且存在备份服务器时,执行步骤504;当该第一存储记录中不包含该待读取数据的数据标识且不存在备份服务器时,且不存在备份服务器时,执行步骤 506。
步骤503:当该第一存储记录中包含该待读取数据的数据标识时,缓存服务器根据该待读取数据的数据标识,从第一存储记录中确定该待读取数据的第一存储路径,根据该待读取数据的第一存储路径,从该第一固态硬盘中读取该待读取数据,执行步骤507。
步骤504:当该第一存储记录中不包含该待读取数据的数据标识时,缓存服务器确定第三存储记录中是否包含该待读取数据的数据标识,该第三存储记录为该缓存服务器的备份服务器的存储记录。
当缓存服务器中存储第三存储记录时,缓存服务器直接确定本地存储的第三存储记录中是否包含该待读取数据的数据标识。当缓存服务器不存储第三存储记录时,缓存服务器确定第三存储记录中是否包含该待读取数据的数据标识的步骤可以为:缓存服务器向备份服务器发送查询请求,该查询请求携带待读取数据的数据标识。备份服务器接收缓存服务器的查询请求,确定第三存储记录中是否包含该待读取数据的数据标识,向缓存服务器返回查询结果,该查询结果为第三存储记录中包含该待读取数据的数据标识,或者第三存储记录中不包含该待读取数据的数据标识。
当第三存储记录中包含该待读取数据的数据标识时,执行步骤505;当第三存储记录中不包含该待读取数据的数据标识时,执行步骤506。
需要说明的一点是,当备份服务器确定第三存储记录中包含该待读取数据的数据标识时,还可以直接根据该待读取数据的数据标识,读取该待读取数据,该查询结果中直接携带该待读取数据,从而进一步提高读取效率。
步骤505:当第三存储记录中包含该待读取数据的数据标识时,缓存服务器根据该待读取数据的数据标识,从该第三存储记录中确定该待读取数据的第三存储路径,根据该待读取数据的第三存储路径,从该备份服务器中的第二固态硬盘中读取该待读取数据,执行步骤507。
第一种实现方式,缓存服务器向备份服务器发送该待读取数据的数据标识,以从备份服务器中的第二固态硬盘中读取该待读取数据。相应的,本步骤可以为:缓存服务器向备份服务器发送第一下载请求,第一下载请求携带该待读取数据的数据标识。备份服务器接收缓存服务器的第一下载请求,根据该待读取数据的数据标识,从该第三存储记录中确定该待读取数据的第三存储路径,根据该待读取数据的第三存储路径,从该备份服务器中的第二固态硬盘中读取该 待读取数据。备份服务器向缓存服务器返回该待读取数据。
在另一个可能的实现方式中,缓存服务器中存储数据标识和第三存储路径的对应关系,缓存服务器可以获取到该待转存数据在第二固态硬盘中的第三存储路径,直接通知备份服务器从第三存储路径对应的存储空间中的该转存数据。相应的,本步骤可以为:
缓存服务器根据该待读取数据的数据标识,从该第三存储记录中确定该待读取数据的第三存储路径,向备份服务器发送第二下载请求,第二下载请求携带该待读取数据的第三存储路径。备份服务器接收缓存服务器的第二下载请求,根据该待读取数据的第三存储路径,从该备份服务器中的第二固态硬盘中读取该待读取数据。备份服务器向缓存服务器返回该待读取数据。
步骤506:当第三存储记录中不包含该待读取数据的数据标识时,缓存服务器根据该待读取数据的数据标识,从该第二存储记录中确定该待读取数据的第二存储路径,根据该待读取数据的第二存储路径,从该云服务器中读取该待读取数据,执行步骤507。
第一种实现方式,缓存服务器向云服务器发送该待读取数据的数据标识,以从云服务器中读取该待读取数据。相应的,本步骤可以为:缓存服务器向云服务器发送第三下载请求,第三下载请求携带该待读取数据的数据标识。云服务器接收缓存服务器的第三下载请求,根据该待读取数据的数据标识,从该第二存储记录中确定该待读取数据的第二存储路径,根据该待读取数据的第二存储路径,从该云服务器中读取该待读取数据。云服务器向缓存服务器返回该待读取数据。
在另一个可能的实现方式中,缓存服务器中存储数据标识和第二存储路径的对应关系,缓存服务器可以获取到该待转存数据在云服务器中的第二存储路径,直接通知云服务器从第二存储路径对应的存储空间中的该转存数据。相应的,本步骤可以为:
缓存服务器根据该待读取数据的数据标识,从该第二存储记录中确定该待读取数据的第二存储路径,向云服务器发送第四下载请求,第四下载请求携带该待读取数据的第二存储路径。云服务器接收缓存服务器的第四下载请求,根据该待读取数据的第二存储路径,从该云服务器中读取该待读取数据。云服务器向缓存服务器返回该待读取数据。
需要说明的一点是,当不存在备份服务器时,在步骤502中确定出第一存 储记录中不包含该待读取数据的数据标识时,执行步骤步骤506,且步骤506可以替换为:当第一存储记录中不包含该待读取数据的数据标识时,缓存服务器根据该待读取数据的数据标识,从该第二存储记录中确定该待读取数据的第二存储路径,根据该待读取数据的第二存储路径,从该云服务器中读取该待读取数据,执行步骤507。
步骤507:缓存服务器向该第二终端返回该待读取数据。
在本申请实施例中,在云存储系统中设置缓存服务器,且缓存服务器中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到新的磁盘中。因此,第二终端在读取数据时,缓存服务器优先从缓存服务器中读取待读取数据。当缓存服务器中不存储该待读取数据时,从备份服务器中读取该待读取数据。当备份服务器中不存储该待读取数据时,才用云服务器中读取该待读取数据。由于缓存服务器和备份服务器均使用固态硬盘存储数据,因此从缓存服务器或者备份服务器中可以快速下载数据,从而可以支持用户数据即存即取的需求。
本申请实施例提供了一种基于云存储的数据存储装置,所述装置应用于缓存服务器,用于执行上述基于云存储的数据存储方法中缓存服务器执行的步骤。参见图6,所述装置包括:
接收模块601,用于接收第一终端发送的存储请求,所述存储请求携带待存储数据;
存储模块602,用于将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;
转存模块603,用于定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
在一个可能的实现方式中,所述转存模块603,还用于定期监测所述第一固态硬盘的空间利用率;当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储 记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
在另一个可能的实现方式中,所述转存模块603,还用于根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
所述转存模块603,还用于根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
在另一个可能的实现方式中,参见图7,所述装置还包括:
备份模块604,用于将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
在另一个可能的实现方式中,参见图8,所述装置还包括:
删除模块605,用于在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
在另一个可能的实现方式中,参见图9,所述装置还包括:
所述接收模块601,还用于接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
读取模块606,用于根据所述待读取数据的数据标识,读取所述待读取数据;
返回模块607,用于向所述第二终端返回所述待读取数据。
在另一个可能的实现方式中,所述读取模块606,还用于确定所述第一存储记录中是否包含所述待读取数据的数据标识;当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
在另一个可能的实现方式中,所述读取模块606,还用于确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;当所述第三存储记录中包含所述待读取数据的数据 标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;当所述第三存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径。
在本申请实施例中,在云存储系统中设置缓存服务器,且缓存服务器中是使用固态硬盘存储数据,而固态硬盘自身特性可以保证数据实时写入并刷新到新的磁盘中。因此,第一终端在存储数据时,缓存服务器优先将待存储数据写入缓存服务器,再由缓存服务器后台异步转存至云服务器进行持久化存储。从而缓存服务器可以快速存储和下载数据,从而可以支持用户数据即存即取的需求。
需要说明的是:上述实施例提供的基于云存储的数据存储装置在基于云存储的数据存储时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的基于云存储的数据存储装置与基于云存储的数据存储方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。
图10是本申请实施例提供的一种缓存服务器的结构示意图,该缓存服务器1000可因配置或性能不同而产生比较大的差异,可以包括一个或一个以上处理器(central processing units,CPU)1001和一个或一个以上的存储器1002,其中,所述存储器1002中存储有至少一条指令,所述至少一条指令由所述处理器1001加载并执行以实现上述各个方法实施例提供的方法。当然,该服务器还可以具有有线或无线网络接口、键盘以及输入输出接口等部件,以便进行输入输出,该服务器还可以包括其他用于实现设备功能的部件,在此不做赘述。
本申请实施例提供了一种非易失性计算机可读存储介质,用于存储计算机程序,所述计算机程序通过处理器进行加载来执行上述基于云存储的数据存储方法的指令。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。
以上所述仅为本申请的较佳实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (18)

  1. 一种基于云存储的数据存储方法,其特征在于,所述方法应用于缓存服务器,所述方法包括:
    接收第一终端发送的存储请求,所述存储请求携带待存储数据;
    将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;
    定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
  2. 根据权利要求1所述的方法,其特征在于,所述定期将所述第一固态硬盘中待转存数据转存至云服务器中,包括:
    定期监测所述第一固态硬盘的空间利用率;
    当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;
    记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
  3. 根据权利要求2所述的方法,其特征在于,所述从所述第一固态硬盘中选择所述待转存数据,包括:
    根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
    根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
  4. 根据权利要求1所述的方法,其特征在于,所述将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储 记录中之后,所述方法还包括:
    将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
  5. 根据权利要求4所述的方法,其特征在于,所述将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中之后,所述方法还包括:
    在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;
    将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
  6. 根据权利要求1-5任一项权利要求所述的方法,其特征在于,所述方法还包括:
    接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
    根据所述待读取数据的数据标识,读取所述待读取数据;
    向所述第二终端返回所述待读取数据。
  7. 根据权利要求6所述的方法,其特征在于,所述根据所述待读取数据的数据标识,读取所述待读取数据,包括:
    确定所述第一存储记录中是否包含所述待读取数据的数据标识;
    当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;
    当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
  8. 根据权利要求7所述的方法,其特征在于,所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径之前,所述方法还包括:
    确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;
    当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;
    当所述第三存储记录中不包含所述待读取数据的数据标识时,执行所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径的步骤。
  9. 一种基于云存储的数据存储装置,其特征在于,所述装置应用于缓存服务器,所述装置包括:
    接收模块,用于接收第一终端发送的存储请求,所述存储请求携带待存储数据;
    存储模块,用于将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;
    转存模块,用于定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
  10. 根据权利要求9所述的装置,其特征在于,
    所述转存模块,还用于定期监测所述第一固态硬盘的空间利用率;当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数 据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
  11. 根据权利要求10所述的装置,其特征在于,
    所述转存模块,还用于根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,
    所述转存模块,还用于根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
  12. 根据权利要求9所述的装置,其特征在于,所述装置还包括:
    备份模块,用于将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
  13. 根据权利要求12所述的装置,其特征在于,所述装置还包括:
    删除模块,用于在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
  14. 根据权利要求9-13任一项权利要求所述的装置,其特征在于,所述装置还包括:
    所述接收模块,还用于接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;
    读取模块,用于根据所述待读取数据的数据标识,读取所述待读取数据;
    返回模块,用于向所述第二终端返回所述待读取数据。
  15. 根据权利要求14所述的装置,其特征在于,
    所述读取模块,还用于确定所述第一存储记录中是否包含所述待读取数据的数据标识;当所述第一存储记录中包含所述待读取数据的数据标识时,根据 所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
  16. 根据权利要求15所述的装置,其特征在于,
    所述读取模块,还用于确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;当所述第三存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径。
  17. 一种基于云存储的数据存储装置,其特征在于,所述装置包括
    至少一个处理器;和
    至少一个存储器;
    所述至少一个存储器存储有一个或多个程序,所述一个或多个程序被配置成由所述至少一个处理器执行,所述一个或多个程序包含用于进行如权利要求1至8任一项权利要求所述的基于云存储的数据存储方法的指令。
  18. 一种非易失性计算机可读存储介质,其特征在于,用于存储计算机程序,所述计算机程序通过处理器进行加载来执行如权利要求1至8任一项权利要求所述的基于云存储的数据存储方法的指令。
PCT/CN2019/117570 2018-11-13 2019-11-12 基于云存储的数据存储方法、装置和存储介质 WO2020098654A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811348900.XA CN111176549B (zh) 2018-11-13 2018-11-13 基于云存储的数据存储方法、装置和存储介质
CN201811348900.X 2018-11-13

Publications (1)

Publication Number Publication Date
WO2020098654A1 true WO2020098654A1 (zh) 2020-05-22

Family

ID=70657337

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/117570 WO2020098654A1 (zh) 2018-11-13 2019-11-12 基于云存储的数据存储方法、装置和存储介质

Country Status (2)

Country Link
CN (1) CN111176549B (zh)
WO (1) WO2020098654A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112596682A (zh) * 2020-12-28 2021-04-02 郝东东 一种区块链用存储装置及存储方法
CN114461146A (zh) * 2022-01-26 2022-05-10 北京百度网讯科技有限公司 云存储的数据处理方法、装置、系统、设备、介质及产品
CN115202312A (zh) * 2022-09-16 2022-10-18 山东神驰化工集团有限公司 一种智能式化工安全生产巡控系统
CN115543185A (zh) * 2022-03-30 2022-12-30 中国移动通信集团四川有限公司 数据存储管理方法及系统、装置、计算机可读存储介质
WO2023060943A1 (zh) * 2021-10-14 2023-04-20 华为技术有限公司 一种流控方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115065694B (zh) * 2022-07-29 2024-02-09 苏州浪潮智能科技有限公司 一种云存储数据中转上传系统、方法、设备及介质
CN116340055B (zh) * 2023-03-28 2024-03-12 上海威固信息技术股份有限公司 一种固态硬盘的数据恢复方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060004957A1 (en) * 2002-09-16 2006-01-05 Hand Leroy C Iii Storage system architectures and multiple caching arrangements
CN103034592A (zh) * 2012-12-05 2013-04-10 华为技术有限公司 数据处理方法和装置
CN103795756A (zh) * 2012-10-31 2014-05-14 中国电信股份有限公司 分层存储方法和系统、对象访问控制器
CN104618482A (zh) * 2015-02-02 2015-05-13 浙江宇视科技有限公司 访问云数据的方法、服务器、传统存储设备、架构
CN105491149A (zh) * 2015-12-26 2016-04-13 深圳市金立通信设备有限公司 一种数据存储方法及终端
CN106911776A (zh) * 2017-02-24 2017-06-30 郑州云海信息技术有限公司 一种云存储设备的管理方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103873501B (zh) * 2012-12-12 2017-07-18 华中科技大学 一种云备份系统及其数据备份方法
CN103605585B (zh) * 2013-11-06 2017-01-18 上海爱数信息技术股份有限公司 一种基于数据发现的智能备份方法
CN106161523B (zh) * 2015-04-02 2019-11-22 腾讯科技(深圳)有限公司 一种数据处理方法和设备
CN107786599B (zh) * 2016-08-26 2020-07-17 北京计算机技术及应用研究所 内存云系统
CN108093207B (zh) * 2016-11-23 2021-03-16 杭州海康威视数字技术股份有限公司 一种数据存储、获取方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060004957A1 (en) * 2002-09-16 2006-01-05 Hand Leroy C Iii Storage system architectures and multiple caching arrangements
CN103795756A (zh) * 2012-10-31 2014-05-14 中国电信股份有限公司 分层存储方法和系统、对象访问控制器
CN103034592A (zh) * 2012-12-05 2013-04-10 华为技术有限公司 数据处理方法和装置
CN104618482A (zh) * 2015-02-02 2015-05-13 浙江宇视科技有限公司 访问云数据的方法、服务器、传统存储设备、架构
CN105491149A (zh) * 2015-12-26 2016-04-13 深圳市金立通信设备有限公司 一种数据存储方法及终端
CN106911776A (zh) * 2017-02-24 2017-06-30 郑州云海信息技术有限公司 一种云存储设备的管理方法及装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112596682A (zh) * 2020-12-28 2021-04-02 郝东东 一种区块链用存储装置及存储方法
WO2023060943A1 (zh) * 2021-10-14 2023-04-20 华为技术有限公司 一种流控方法及装置
CN114461146A (zh) * 2022-01-26 2022-05-10 北京百度网讯科技有限公司 云存储的数据处理方法、装置、系统、设备、介质及产品
CN114461146B (zh) * 2022-01-26 2024-05-07 北京百度网讯科技有限公司 云存储的数据处理方法、装置、系统、设备、介质及产品
CN115543185A (zh) * 2022-03-30 2022-12-30 中国移动通信集团四川有限公司 数据存储管理方法及系统、装置、计算机可读存储介质
CN115543185B (zh) * 2022-03-30 2023-12-19 中国移动通信集团四川有限公司 数据存储管理方法及系统、装置、计算机可读存储介质
CN115202312A (zh) * 2022-09-16 2022-10-18 山东神驰化工集团有限公司 一种智能式化工安全生产巡控系统
CN115202312B (zh) * 2022-09-16 2022-12-27 山东神驰化工集团有限公司 一种智能式化工安全生产巡控系统

Also Published As

Publication number Publication date
CN111176549A (zh) 2020-05-19
CN111176549B (zh) 2021-05-18

Similar Documents

Publication Publication Date Title
WO2020098654A1 (zh) 基于云存储的数据存储方法、装置和存储介质
US9430321B2 (en) Reconstructing data stored across archival data storage devices
US9436571B2 (en) Estimating data storage device lifespan
US20220035555A1 (en) Index Lifecycle Management
US8615485B2 (en) Method and system for managing weakly mutable data in a distributed storage system
US9189493B2 (en) Object file system
US20150356125A1 (en) Method for data placement based on a file level operation
US20150254320A1 (en) Using colocation hints to facilitate accessing a distributed data storage system
WO2019001521A1 (zh) 数据存储方法、存储设备、客户端及系统
US9557938B2 (en) Data retrieval based on storage device activation schedules
CN103917962A (zh) 读取存储在存储系统上的文件
US20140281301A1 (en) Elastic hierarchical data storage backend
US9436524B2 (en) Managing archival storage
US20130325932A1 (en) Electronic device and method for storing distributed documents
CN115202929B (zh) 数据库集群备份系统
JP2015527620A (ja) 計算機システム、サーバ、及び、データ管理方法
CN110502187B (zh) 一种快照回滚方法及装置
WO2014153931A1 (zh) 文件存储方法、装置、访问客户端及元数据服务器系统
US9430149B2 (en) Pipeline planning for low latency storage system
US11157511B2 (en) Physical replication of database
CN109144403B (zh) 一种用于云盘模式切换的方法与设备
US9430152B2 (en) Data device grouping across data storage device enclosures for synchronized data maintenance
US9766677B2 (en) Cascading startup power draws of enclosures across a network
CN112286448B (zh) 对象访问方法、装置、电子设备及机器可读存储介质
WO2018077092A1 (zh) 应用于分布式文件系统的存盘方法、装置及分布式文件系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19883815

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19883815

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19883815

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 04/02/2022)