WO2020098654A1 - 基于云存储的数据存储方法、装置和存储介质 - Google Patents
基于云存储的数据存储方法、装置和存储介质 Download PDFInfo
- Publication number
- WO2020098654A1 WO2020098654A1 PCT/CN2019/117570 CN2019117570W WO2020098654A1 WO 2020098654 A1 WO2020098654 A1 WO 2020098654A1 CN 2019117570 W CN2019117570 W CN 2019117570W WO 2020098654 A1 WO2020098654 A1 WO 2020098654A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- storage
- read
- solid
- identifier
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000013500 data storage Methods 0.000 title claims abstract description 34
- 239000007787 solid Substances 0.000 claims abstract description 111
- 238000012546 transfer Methods 0.000 claims abstract description 33
- 238000012217 deletion Methods 0.000 claims description 9
- 230000037430 deletion Effects 0.000 claims description 9
- 230000015654 memory Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002085 persistent effect Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 238000013403 standard screening design Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001934 delay Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Definitions
- the present application relates to the field of cloud storage technology, and in particular, to a data storage method, device, and storage medium based on cloud storage.
- Cloud storage has gradually emerged with its massive storage space, high reliability, high concurrency, high scalability and high cost performance. Therefore, more and more users choose to store data in cloud servers.
- cloud servers generally use mechanical hard disks to store data.
- the cloud server receives the data to be stored, it stores the data to be stored in the mechanical hard disk, and then refreshes the data in the mechanical hard disk to the physical hard disk.
- This application provides a cloud storage-based data storage method, device, and storage medium, which can solve the problem that the cloud server cannot support the user's demand for data to be stored and retrieved.
- the technical solution is as follows:
- the embodiments of the present application provide a cloud storage-based data storage method.
- the method is applied to a cache server.
- the method includes:
- the periodically dumping the data to be dumped in the first solid state disk to the cloud server includes:
- the selecting the data to be dumped from the first solid state drive includes:
- the storage time of the data in the first solid state drive select the data with the furthest storage time away from the current time as the data to be transferred from the first solid state drive; or,
- the data with the smallest thermal value is selected from the first solid state drive as the data to be dumped, and the thermal value of the data is used to indicate the probability of the data being read.
- the method further include:
- the method further includes :
- the method further includes:
- the reading the data to be read according to the data identifier of the data to be read includes:
- the first storage of the data to be read is determined from the first storage record according to the data identifier of the data to be read Path, according to the first storage path of the data to be read, reading the data to be read from the first solid state drive;
- the second identifier of the data to be read is determined from the second storage record according to the data identifier of the data to be read
- the storage path reads the data to be read from the cloud server according to the second storage path of the data to be read.
- the method before determining the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, the method further includes:
- the third storage record contains the data identifier of the data to be read, and the third storage record is a storage record of the backup server of the cache server;
- the third storage of the data to be read is determined from the third storage record according to the data identification of the data to be read Path, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server;
- an embodiment of the present application provides a data storage device based on cloud storage.
- the device is applied to a cache server.
- the device includes:
- a receiving module configured to receive a storage request sent by the first terminal, where the storage request carries data to be stored
- a storage module configured to store the data to be stored in the first solid-state hard disk in the cache server, and to correspond the data identification of the data to be stored to its first storage path in the first solid-state hard disk The relationship is stored in the first storage record;
- the dump module is used to periodically dump the data to be dumped in the first solid-state hard disk to the cloud server, and to correspond the data identifier of the data to be dumped to the second storage path in the cloud server The relationship is stored in the second storage record.
- the dump module is also used to periodically monitor the space utilization rate of the first solid state drive; when the space utilization rate is greater than the specified utilization rate, from the first solid state drive Select the data to be transferred and store the data to be transferred to the cloud server; record the transfer time of the data to be transferred, the difference between the current time and the transfer time exceeds
- the data to be dumped is deleted from the first solid state hard disk, and the correspondence between the data identifier of the data to be dumped in the first storage record and the first storage path is deleted.
- the dump module is further configured to select, from the first solid-state hard disk, data whose storage time is the farthest from the current time as the data of the first solid-state hard disk as the storage time Data to be transferred; or,
- the dumping module is further configured to select the data with the smallest thermal value from the first solid-state hard disk as the data to be dumped according to the thermal value of the data in the first solid-state hard disk, and the thermal value of the data is used to indicate The probability of the data being read.
- the device further includes:
- a backup module configured to back up the data to be stored to the second solid-state hard disk in the backup server, and store the correspondence between the data identification of the data to be stored and the third storage path in the second solid-state hard disk In the third storage record.
- the device further includes:
- a deletion module configured to delete the data to be transferred from the second solid-state drive when the difference between the current time and the transfer time of the data to be transferred exceeds a second specified duration; The corresponding relationship between the data identifier of the data to be dumped and the third storage path in the third storage record is deleted.
- the device further includes:
- the receiving module is also used to receive a read request sent by the second terminal, where the read request carries a data identifier of the data to be read;
- a reading module configured to read the data to be read according to the data identifier of the data to be read
- the return module is configured to return the data to be read to the second terminal.
- the reading module is further configured to determine whether the data identifier of the data to be read is included in the first storage record; when the first storage record includes the data identifier When reading the data identifier of the data, determine the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read, and according to the first of the data to be read A storage path to read the data to be read from the first solid-state hard disk; when the first storage record does not include the data identifier of the data to be read, according to the data of the data to be read Mark, determine the second storage path of the data to be read from the second storage record, and read the data to be read from the cloud server according to the second storage path of the data to be read .
- the reading module is further configured to determine whether a data identifier of the data to be read is included in a third storage record, and the third storage record is a backup server of the cache server Storage record; when the third storage record contains the data identifier of the data to be read, determine the data to be read from the third storage record according to the data identifier of the data to be read.
- the third storage path according to the third storage path of the data to be read, read the data to be read from the second SSD in the backup server; when the third storage record does not contain
- the second storage path of the data to be read is determined from the second storage record according to the data identification of the data to be read.
- an embodiment of the present application provides a data storage device based on cloud storage, the device includes
- At least one processor At least one processor
- At least one memory At least one memory
- the at least one memory stores one or more programs, the one or more programs are configured to be executed by the at least one processor, and the one or more programs are included in any possible implementation manner described above
- the instructions of the data storage method based on cloud storage.
- an embodiment of the present application provides a non-volatile computer-readable storage medium for storing a computer program, the computer program is loaded by a processor to execute as described in any possible implementation manner described above Instructions for data storage methods based on cloud storage.
- a cache server is provided in the cloud storage system, and a solid-state drive is used to store data in the cache server, and the characteristics of the solid-state drive can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
- FIG. 1 is a schematic structural diagram of a cloud storage system provided by an embodiment of the present application.
- FIG. 2 is a schematic structural diagram of another cloud storage system provided by an embodiment of the present application.
- FIG. 3 is a method flowchart of a data storage method based on cloud storage provided by an embodiment of the present application
- FIG. 4 is a method flowchart of a data storage method based on cloud storage provided by an embodiment of the present application
- FIG. 5 is a method flowchart of a cloud storage-based data storage method provided by an embodiment of the present application.
- FIG. 6 is a schematic structural diagram of a cloud storage data storage device provided by an embodiment of the present application.
- FIG. 7 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
- FIG. 8 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
- FIG. 9 is a schematic structural diagram of another cloud storage data storage device provided by an embodiment of the present application.
- FIG. 10 is a schematic structural diagram of a cache server provided by an embodiment of the present application.
- FIG. 1 is a cloud storage system provided by an embodiment of the present application.
- the cloud storage system includes: a first terminal 101, a second terminal 102, a cache server 103, and a cloud server 104.
- the first terminal 101 and the second terminal 102 are respectively connected to the cache server 103 through a network, and the cache server 103 and the cloud server 104 are connected through a network.
- the first terminal 101 is a terminal that stores data.
- the second terminal 102 is a terminal that reads data.
- the first terminal 101 and the second terminal 102 may be the same terminal or different terminals.
- the cloud server 104 may be a cloud storage system.
- the first terminal 101 is used to upload data to be stored to the cache server 103.
- the cache server 103 is used to store the data to be stored in the first solid-state hard disk included in it, and store the correspondence between the data identification of the data to be stored and its first storage path in the first solid-state hard disk in the first storage record in.
- the cache server 103 is also used to periodically transfer the data to be transferred from the first solid-state drive to the cloud server 104, and store the correspondence between the data identifier of the data to be transferred and the second storage path in the cloud server 104 in the first Two storage records.
- the cache server 103 transfers the data to be transferred from the first solid-state drive to the cloud server 104, and the data to be transferred may be storage Old data with a long period of time or data with relatively low heat.
- the cache server 103 deletes the data to be transferred from the first solid-state drive after a period of time after storing the data to be transferred into the first solid-state drive.
- the cloud server 104 since the cloud server 104 generally uses an ordinary mechanical hard disk, after the cache server 103 transfers the data to be dumped to the cloud server 104, the data to be stored in the first solid-state hard disk cannot be deleted immediately, to avoid the data of the mechanical hard disk of the cloud server 104 Failure to implement refresh orders will take effect.
- the cache server 103 deletes the data to be transferred from the first solid-state hard disk, and releases the storage space in the first solid-state hard disk, so as to continuously cache the newly uploaded data of the first terminal 101.
- the second terminal 102 is used to download the data to be read to the cache server 103.
- the cache server 103 determines the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read, according to the The first storage path of the data to be read reads the data to be read from the first solid-state hard disk.
- the cache server 103 determines the second storage path of the data to be read from the second storage record according to the data identifier according to the data to be read, and according to the data to be read The second storage path, read the data to be read from the cloud server 104.
- a cache server 103 is provided in the cloud storage system, and a solid-state drive is used to store data in the cache server 103, and the characteristics of the solid-state drive can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal 101 stores data, the cache server 103 preferentially writes the data to be stored into the cache server 103, and then the cache server 103 transfers the data asynchronously to the cloud server 104 for persistent storage. Therefore, the cache server 103 can quickly store and download data, and thus can support the user's demand for data to be stored and retrieved.
- the cloud storage system further includes a backup server 105, and the backup server 105 and the cache server 103 are connected through a network.
- the cache server 103 stores the data to be stored in the first solid-state hard disk included in it
- the data to be stored is stored in the second solid-state hard disk included in the backup server 105, thereby further improving the security of the data to be stored.
- the cache server 103 deletes the data to be stored from the first solid-state drive
- the data to be stored is also deleted from the second solid-state drive, so that the second solid-state drive can continue to cache newly uploaded data from the first terminal 101 data.
- An embodiment of the present application provides a data storage method based on cloud storage, and the method is applied to a cache server.
- the method includes:
- Step 301 Receive a storage request sent by a first terminal, where the storage request carries data to be stored.
- Step 302 Store the data to be stored in the first solid state drive in the cache server, and store the correspondence between the data identifier of the data to be stored and its first storage path in the first solid state drive In the first storage record.
- Step 303 Periodically transfer the data to be dumped in the first solid-state hard disk to the cloud server, and store the correspondence between the data identifier of the data to be dumped and its second storage path in the cloud server in In the second storage record.
- the periodically dumping the data to be dumped in the first solid state disk to the cloud server includes:
- the selecting the data to be dumped from the first solid state drive includes:
- the storage time of the data in the first solid state drive select the data with the furthest storage time away from the current time as the data to be transferred from the first solid state drive; or,
- the data with the smallest thermal value is selected from the first solid state drive as the data to be dumped, and the thermal value of the data is used to indicate the probability of the data being read.
- the method further include:
- the method further includes :
- the method further includes:
- the reading the data to be read according to the data identifier of the data to be read includes:
- the first storage of the data to be read is determined from the first storage record according to the data identifier of the data to be read Path, according to the first storage path of the data to be read, reading the data to be read from the first solid state drive;
- the second identifier of the data to be read is determined from the second storage record according to the data identifier of the data to be read
- the storage path reads the data to be read from the cloud server according to the second storage path of the data to be read.
- the method before determining the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, the method further includes:
- the third storage record contains the data identifier of the data to be read, and the third storage record is a storage record of the backup server of the cache server;
- the third storage of the data to be read is determined from the third storage record according to the data identification of the data to be read Path, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server;
- a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
- An embodiment of the present application provides a data storage method based on cloud storage.
- a first terminal is used to store data as an example for illustration.
- the method is applied between a first terminal, a cache server, a cloud server, and a backup server. .
- the method includes:
- Step 401 The first terminal sends a storage request to the cache server, where the storage request carries data to be stored.
- the first terminal runs a cloud storage application, logs in to the cache server based on the cloud storage application, and sends a storage request to the cache server, where the storage request carries data to be stored.
- the data to be stored may be data in any format.
- the data to be stored may be text, audio data, video data, pictures, and so on.
- the data to be stored may be data already stored by the first terminal, or may be data currently received by the first terminal; the data to be stored may also be a chat record of a social application, or data in a specified format in the social application ( Pictures or videos, etc.).
- the data to be stored when the data to be stored is a picture, the data to be stored may be a picture that has been stored in the picture library, or a picture that is currently taken.
- the data to be stored is a picture
- the data to be stored may be a picture currently received through a social application.
- the heat value of the data to be stored is carried in the storage request, but also the heat value of the data to be stored may be carried in the storage request.
- the heat value of the data to be stored is used to indicate the probability of the data to be stored being read.
- the first terminal may upload the data to be stored to the cache server when the user triggers the storage instruction, which is the first implementation manner.
- the first terminal can also automatically upload the unuploaded data to be stored to the cache server, that is, the second implementation manner.
- this step may be: the first terminal receives the storage instruction, displays an upload interface, and the upload interface includes a data index of multiple data.
- the first terminal acquires the selected data index, acquires data corresponding to the data index based on the data index, and uses the acquired data as data to be stored.
- the upload interface also includes an upload button; when the first terminal detects that the upload button is triggered, it sends a storage request to the cache server, where the storage request carries the data to be stored.
- this step may be: the first terminal obtains unuploaded data to be stored from the database, and sends a storage request to the cache server, where the storage request carries the data to be stored.
- the database may be a picture library.
- the first terminal can automatically identify the unuploaded data to be stored, automatically upload the unuploaded data to the cache server, and then store it in the cloud server, and the production can be realized without the participation of the user. , Not only improve upload efficiency, but also increase user viscosity.
- the first terminal needs to consume more network resources to send the storage request to the cache server.
- the first terminal determines the network type that the first terminal is currently connected to; when the network type is a wireless network, the step of sending the storage request from the first terminal to the cache server is performed; when the network type When it is a mobile network, the first terminal temporarily does not send a storage request to the cache server, and only sends a storage request to the cache server when it detects a wireless network.
- Step 402 The cache server receives the storage request of the first terminal, stores the data to be stored in the first solid state drive, and the correspondence between the data identifier of the data to be stored and its first storage path in the first solid state drive Stored in the first storage record.
- the cache server includes a first solid state drive.
- the cache server stores the data to be stored carried in the storage request in the first solid state drive, determines the first storage path of the data to be stored in the first solid state drive, and identifies the data of the data to be stored The corresponding relationship with the first storage path is stored in the first storage record.
- the cache server may include one solid state drive or multiple solid state drives; when the cache server includes one solid state drive, the cache server may directly store the data to be stored in the one solid state drive; when the cache server includes multiple In the case of a solid-state drive, the cache server may select one solid-state drive from multiple solid-state drives, and store the data to be stored in the selected solid-state drive.
- the cache server when the cache server selects the solid-state drive, the cache server can randomly select one solid-state drive from multiple solid-state drives; it can also select a solid-state drive with the lowest space utilization rate from the multiple solid-state drives; Priority level and space utilization of each solid state drive, select a solid state drive that matches the priority level from multiple solid state drives. For example, when the priority level of the first terminal is high, select a solid state drive with a low space utilization rate from multiple solid state drives; when the priority level of the first terminal is low, select a space utilization rate from multiple solid state drives Higher solid state drive.
- the cache server can also select a solid-state drive that matches the thermal value from multiple solid-state drives according to the thermal value of the data to be stored and the space utilization of each solid-state drive; for example, when the thermal value of the data to be stored is high When selecting the SSD with high space utilization from multiple SSDs; when the heat value of the data to be stored is low, select the SSD with low space utilization from among multiple SSDs.
- the cache server after the cache server stores the data to be stored in the first solid-state disk, the storage time of the data to be stored is obtained, and the data identifier and storage time of the data to be stored are associated to facilitate subsequent Data storage data transfers the data.
- the cache server when the storage request also carries the heat value of the data to be stored, the cache server also stores the correspondence between the data identifier of the data to be stored and the heat value, so as to subsequently obtain the heat value of the data to be stored.
- a backup server may be set for the cache server. Therefore, after step 402 is executed, step 403 is executed, and then step 404 is executed. In addition, when no backup server is provided for the cache server, after step 402 is executed, step 404 is directly executed.
- the cache server When the cache server stores the data to be stored in the first solid-state hard disk, it will return a successful storage response to the first terminal, where the successful storage response is used to indicate that the data to be stored has been successfully stored.
- the successful storage response is used to indicate that the data to be stored has been successfully stored.
- the storage speed of the first SSD is extremely fast. Therefore, storing the data to be stored in the first solid-state hard disk returns a successful storage response to the first terminal. For the user, the response time is shortened and the user viscosity is increased.
- Step 403 The cache server backs up the data to be stored to the second solid state drive in the backup server, and stores the correspondence between the data identifier of the data to be stored and its third storage path in the second solid state drive in the third Store records.
- the cache server sends a backup request to the backup server, where the backup request carries the data to be stored.
- the backup server receives the backup request sent by the cache server, stores the data to be stored in the second solid-state drive included therein, determines a third storage path of the data to be stored in the second solid-state drive, and stores the data of the data to be stored The correspondence between the identifier and the third storage path is stored in the third storage record.
- the backup server may return the third storage record to the cache server.
- the cache server receives the third storage record returned by the backup server.
- the backup server may return the third storage path to the cache server.
- the cache server stores the correspondence between the data identifier of the data to be stored and the third storage path in the third storage record, so as to facilitate subsequent data location query.
- the cache server when the cache server stores the data to be stored in the first solid state drive, it can back up the data to be stored to the second solid state drive in the backup server; the cache server can also periodically store the first The data in the solid state drive is backed up to the second solid state drive in the backup server.
- the cache server may back up all data in the first solid state drive to the backup server, or may only back up more important data to the backup server.
- the cache server determines the importance of the data to be stored; only when the importance of the data to be stored exceeds the second preset threshold. The step of the cache server backing up the data to be stored to the first solid-state hard disk in the backup server is performed; when the important program of the data to be stored does not exceed the first preset threshold, the cache server does not perform the backup operation.
- the step of the cache server determining the importance of the data to be stored may be: the cache server may use the heat value of the data to be stored as the importance of the data to be stored.
- the cache server may also determine the importance of the data to be stored according to the content of the data to be stored; for example, when the content of the data to be stored is a bank bill, determine the importance of the data to be stored to a large value; When the content of the data to be stored is image information without persons, it is determined that the importance of the data to be stored is a small value.
- Step 404 The cache server periodically transfers the data to be transferred from the first solid-state drive to the cloud server, and stores the correspondence between the data identifier of the data to be transferred and its second storage path in the cloud server in the second storage record in.
- the cache server regularly monitors the space utilization rate of the first solid-state drive, and performs the dump processing only when the space utilization rate is high, so as to store the data in the first solid-state drive as much as possible, so that subsequent The data can be directly read from the first solid-state hard disk, which improves the reading efficiency.
- this step can be achieved by the following steps (1) to (4), including:
- the cache server regularly monitors the space utilization of the first solid state drive.
- the cache server monitors the space utilization rate of the first solid-state drive every third specified time period to determine whether the space utilization rate is greater than the specified utilization rate; when the space utilization rate is greater than the specified utilization rate, step (2) is executed; when the space When the utilization rate is not greater than the specified utilization rate, continue to perform step (1) until the space utilization rate of the first solid state drive is greater than the specified utilization rate.
- the third specified duration and specified utilization can be set and changed as needed.
- the third specified duration and specified utilization are not specifically limited; for example, the third specified duration can be 12 hours or 1 Wait.
- the specified utilization rate can be 85% or 90%.
- the cache server selects the data to be transferred from the first solid state disk.
- step (2) can be: the cache server determines the storage data in the first solid state drive, and according to the storage time of the data in the first solid state drive, from The first solid-state hard disk selects the data whose storage time is farthest from the current time, and uses the selected data as the data to be transferred. or,
- the cache server determines the data stored in the first solid-state drive, and selects the data whose time difference between the storage time and the current time reaches the fifth specified duration from the first solid-state drive according to the storage time of the data in the first solid-state drive. Time is used as data to be transferred.
- the cache server has stored the storage time of the data in the first solid state disk.
- the step of the cache server determining the stored data of the data in the first solid state drive may be: the cache server determines the data in the first solid state drive from the correspondence between the data identification and the storage time according to the data identification of the data in the first solid state drive Storage time.
- step (2) may be: the cache server determines the heat value of the data in the first solid state drive, and according to the heat value of the data in the first solid state drive, selects the data with the smallest heat value from the first solid state drive to be transferred data.
- the heat value of the data is used to indicate the probability of the data being read. or,
- the cache server determines the heat value of the data in the first solid state drive, and selects data with a heat value less than the second preset threshold from the first solid state drive as the data to be dumped according to the heat value of the data in the first solid state drive.
- the cache server has stored the correspondence between the data identifier of the data and the heat value.
- the step of the cache server determining the heat value of the data in the first solid state drive may be: according to the data identifier of the data to be dumped, the cache server obtains the heat of the data to be dumped from the corresponding relationship between the data identifier and the heat value value.
- the cache server records the transfer time of the data to be transferred, and deletes the data to be transferred from the first solid state drive when the difference between the current time and the transfer time exceeds the first specified duration Delete the corresponding relationship between the data identifier of the data to be dumped in the first storage record and the first storage path.
- the cache server records the correspondence between the data identifier in the first solid state drive and the first storage path; correspondingly, the cache server may delete the data to be transferred from the first solid state drive as follows:
- the data identifier of the data, the first storage path of the data to be dumped is obtained from the correspondence between the data identifier and the first storage path; according to the first storage path of the data to be dumped, the storage space corresponding to the first storage path Delete the data to be transferred.
- the cache server stores the data to be dumped in the cloud server, it regularly inspects the data that has been dumped in the first solid-state drive, and when the current time is longer than the transfer-out time by more than the first specified time, Only after deleting the data in the first solid-state hard disk, the storage space in the first solid-state hard disk is released, so as to continuously cache the newly uploaded data of the user.
- the first designated duration can be set and changed as needed. In the embodiments of the present application, the first designated duration is not specifically limited; for example, the first designated duration can be 5 minutes or 10 minutes.
- the cloud server usually uses ordinary mechanical hard disks to store data. Therefore, after storing the data to be transferred to the cloud server, the cache server does not immediately delete the data to be transferred in the first solid state drive, but delays deleting the data to be transferred in the first solid state drive, thereby avoiding the cloud server
- the mechanical hard disk has not been refreshed and placed in real time to take effect.
- the cache server stores the data to be dumped in the cloud server, and stores the correspondence between the data identifier of the data to be dumped and its second storage path in the cloud server in the second storage record.
- the cache server sends the data to be transferred to the cloud server.
- the cloud server receives the data to be transferred sent by the cache server, stores the data to be transferred, determines the second storage path of the data to be transferred in the cloud server, and identifies the data identifier and the second storage path of the data to be transferred The corresponding relationship is stored in the second storage record.
- the cloud storage server may send the second storage record to the cache server.
- the cloud server receives the second storage record and stores the second storage record.
- the cloud server may return the second storage path to the cache server.
- the cache server stores the correspondence between the data identifier of the data to be stored and the second storage path in the second storage record, so as to facilitate subsequent data location query.
- the cache server may not perform the transfer process until the space utilization rate in the first solid-state drive reaches the specified utilization rate, but instead transfers the first solid-state drive to be transferred every fourth specified duration
- the stored data is transferred to the cloud server.
- the fourth designated duration can be set and changed as needed. In the embodiments of the present application, the fourth designated duration is not specifically limited; for example, the fourth designated duration can be 1 day or 2 days.
- step 405 is executed to notify the backup server to delete the same data.
- Step 405 When the difference between the current time and the transfer time of the data to be transferred exceeds the second specified duration, the cache server deletes the data to be transferred from the second solid-state hard disk and stores the third The corresponding relationship between the data identifier of the data to be transferred and the third storage path in the record is deleted.
- the cache server sends the data identifier of the data to be dumped to the backup server to notify the backup server to delete the data to be dumped from the second solid-state hard disk.
- the step of the cache server deleting the data to be dumped from the second solid state disk may be: the cache server sends a first deletion request to the backup server, and the first deletion request carries the data identifier of the dumped data.
- the backup server receives the first deletion request of the cache server, obtains the third storage path of the data to be transferred from the correspondence between the data identification and the third storage path according to the data identification of the data to be transferred, and stores the third storage path according to the data to be transferred
- the third storage path of the data deletes the data to be transferred from the storage space corresponding to the third storage path.
- the corresponding relationship between the data identifier and the third storage path is stored in the cache server, and the cache server may obtain the third storage path of the data to be dumped in the second solid-state drive and directly notify the backup server Delete the dumped data in the storage space corresponding to the third storage path.
- the step of the cache server deleting the data to be dumped from the second solid state disk may be: the cache server sends a second deletion request to the backup server, and the second deletion request carries the data identifier of the dumped data.
- the backup server receives the second deletion request of the cache server, and deletes the data to be transferred in the storage space corresponding to the third storage path according to the third storage path of the data to be transferred.
- the cache server deletes the data to be transferred in the first solid-state drive
- the data in the second solid-state drive is deleted and the second solid-state drive is released only when the current time exceeds the transfer-out time by a second specified time
- the second specified duration can be set and changed as needed.
- the second specified duration is not specifically limited; for example, the second specified duration can be 5 minutes or 10 minutes.
- a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
- An embodiment of the present application provides a data storage method based on cloud storage.
- the second terminal reads data as an example for description.
- the method is applied between the second terminal, cache server, cloud server and backup server. Referring to Figure 5, the method includes:
- Step 501 The second terminal sends a read request to the cache server, where the read request carries the data identifier of the data to be read.
- Run a cloud storage application in the second terminal log in to the cache server based on the cloud storage application, and when the second terminal downloads data from the cloud server, the second terminal sends a read request to the cache server, the read request carries a pending read The data identification of the data.
- the first solid-state drive used in the cache server stores data
- the first solid-state drive supports instant storage and retrieval
- the cache server receives the read request, it first determines whether the data to be read is stored in the first solid-state drive ; If the data to be read is stored in the first solid state drive, the data to be read is read from the first solid state drive.
- the cache server determines whether the data to be read is stored in the second solid state drive in the backup server. If the data to be read is stored in the second solid state drive, the data to be read is read from the second solid state drive. If the data to be read is not stored in the second solid-state hard disk, the data to be read is read from the cloud server.
- the cache server determines to read the data to be read directly from the cloud server.
- Step 502 The cache server receives the read request sent by the second terminal, and determines whether the data identifier of the data to be read is included in the first storage record.
- the first storage record includes the correspondence between the data stored in the first solid-state hard disk in the cache server and the first storage path. Therefore, the cache server may determine whether the data to be read is stored in the first solid state drive by determining whether the data identifier of the data to be read is included in the first storage record.
- step 503 When the data identifier of the data to be read is included in the first storage record, it is determined that the data to be read is stored in the first solid state drive, and step 503 is performed; when the first storage record does not contain the data to be read If there is a data identifier of the server and there is a backup server, go to step 504; when the first storage record does not contain the data identifier of the data to be read and there is no backup server, and there is no backup server, go to step 506.
- Step 503 When the first storage record includes the data identifier of the data to be read, the cache server determines the first storage path of the data to be read from the first storage record according to the data identifier of the data to be read , According to the first storage path of the data to be read, read the data to be read from the first solid state drive, and step 507 is executed.
- Step 504 When the first storage record does not contain the data identification of the data to be read, the cache server determines whether the third storage record contains the data identification of the data to be read, and the third storage record is the cache server Storage records of the backup server.
- the cache server directly determines whether the data identifier of the data to be read is included in the locally stored third storage record.
- the step of the cache server determining whether the data identifier of the data to be read is included in the third storage record may be: the cache server sends a query request to the backup server, the query request carrying the query The data identification of the data.
- the backup server receives the query request from the cache server, determines whether the data identifier of the data to be read is contained in the third storage record, and returns a query result to the cache server, which is the data containing the data to be read from the third storage record The identifier, or the third storage record does not contain the data identifier of the data to be read.
- step 505 When the data identifier of the data to be read is included in the third storage record, step 505 is performed; when the data identifier of the data to be read is not included in the third storage record, step 506 is performed.
- the backup server determines that the data identifier of the data to be read is contained in the third storage record, it can also directly read the data to be read according to the data identifier of the data to be read, and the query result Directly carry the data to be read, thereby further improving the reading efficiency.
- Step 505 When the third storage record contains the data identifier of the data to be read, the cache server determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read , According to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server, and step 507 is executed.
- the cache server sends the data identifier of the data to be read to the backup server to read the data to be read from the second solid-state hard disk in the backup server.
- this step may be: the cache server sends a first download request to the backup server, and the first download request carries the data identifier of the data to be read.
- the backup server receives the first download request of the cache server, determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read, and according to the third storage of the data to be read Path to read the data to be read from the second solid-state hard disk in the backup server.
- the backup server returns the data to be read to the cache server.
- the corresponding relationship between the data identifier and the third storage path is stored in the cache server, and the cache server may obtain the third storage path of the data to be dumped in the second solid-state drive and directly notify the backup server The dumped data from the storage space corresponding to the third storage path.
- this step can be:
- the cache server determines the third storage path of the data to be read from the third storage record according to the data identifier of the data to be read, and sends a second download request to the backup server, where the second download request carries the data to be read Third storage path.
- the backup server receives the second download request of the cache server, and reads the data to be read from the second solid-state hard disk in the backup server according to the third storage path of the data to be read.
- the backup server returns the data to be read to the cache server.
- Step 506 When the third storage record does not contain the data identifier of the data to be read, the cache server determines the second storage of the data to be read from the second storage record according to the data identifier of the data to be read Path, read the data to be read from the cloud server according to the second storage path of the data to be read, and perform step 507.
- the cache server sends the data identifier of the data to be read to the cloud server to read the data to be read from the cloud server.
- this step may be: the cache server sends a third download request to the cloud server, and the third download request carries the data identifier of the data to be read.
- the cloud server receives the third download request of the cache server, determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and according to the second storage of the data to be read Path to read the data to be read from the cloud server.
- the cloud server returns the data to be read to the cache server.
- the corresponding relationship between the data identifier and the second storage path is stored in the cache server, and the cache server can obtain the second storage path of the data to be transferred in the cloud server, and directly notify the cloud server to The dumped data in the storage space corresponding to the second storage path.
- this step can be:
- the cache server determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and sends a fourth download request to the cloud server, where the fourth download request carries the data to be read Second storage path.
- the cloud server receives the fourth download request of the cache server, and reads the data to be read from the cloud server according to the second storage path of the data to be read.
- the cloud server returns the data to be read to the cache server.
- step 506 when there is no backup server and it is determined in step 502 that the first storage record does not contain the data identifier of the data to be read, step 506 is performed, and step 506 can be replaced by: When a storage record does not contain the data identifier of the data to be read, the cache server determines the second storage path of the data to be read from the second storage record according to the data identifier of the data to be read, and Read the second storage path of the data, read the data to be read from the cloud server, and perform step 507.
- Step 507 The cache server returns the data to be read to the second terminal.
- a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the second terminal reads the data, the cache server preferentially reads the data to be read from the cache server. When the cache server does not store the data to be read, the data to be read is read from the backup server. When the data to be read is not stored in the backup server, the cloud server is used to read the data to be read. Since both the cache server and the backup server use solid-state hard disks to store data, data can be quickly downloaded from the cache server or the backup server, thereby supporting the user's demand for data to be stored and retrieved.
- An embodiment of the present application provides a cloud storage-based data storage device.
- the device is applied to a cache server, and is used to perform the steps performed by the cache server in the cloud storage-based data storage method.
- the device includes:
- the receiving module 601 is configured to receive a storage request sent by a first terminal, where the storage request carries data to be stored;
- the storage module 602 is configured to store the data to be stored in the first solid state drive in the cache server, and identify the data of the data to be stored and its first storage path in the first solid state drive The corresponding relationship is stored in the first storage record;
- the dump module 603 is used to periodically dump the data to be dumped in the first solid-state hard disk to the cloud server, and to identify the data identifier of the data to be dumped and its second storage path in the cloud server The correspondence is stored in the second storage record.
- the dump module 603 is further used to periodically monitor the space utilization rate of the first solid state drive; when the space utilization rate is greater than the specified utilization rate, the first solid state drive Select the data to be transferred, and store the data to be transferred to the cloud server; record the transfer time of the data to be transferred, the difference between the current time and the transfer time When the first specified time period is exceeded, the data to be dumped is deleted from the first solid state drive, and the correspondence between the data identifier of the data to be dumped in the first storage record and the first storage path is deleted .
- the dump module 603 is further configured to select the data whose storage time is farthest from the current time from the first solid-state drive according to the storage time of the data in the first solid-state drive As data to be transferred; or,
- the dump module 603 is further configured to select the data with the smallest thermal value from the first solid-state hard disk as the data to be dumped according to the thermal value of the data in the first solid-state hard disk, and the thermal value of the data is used to indicate The probability of the data being read.
- the device further includes:
- the backup module 604 is used for backing up the data to be stored to the second solid-state hard disk in the backup server, and mapping the data identification of the data to be stored to its third storage path in the second solid-state hard disk Stored in the third storage record.
- the device further includes:
- the deletion module 605 is configured to delete the data to be transferred from the second solid-state drive when the difference between the current time and the transfer time of the data to be transferred exceeds a second specified duration; The corresponding relationship between the data identifier of the data to be dumped and the third storage path in the third storage record is deleted.
- the device further includes:
- the receiving module 601 is further configured to receive a read request sent by a second terminal, where the read request carries a data identifier of data to be read;
- a reading module 606, configured to read the data to be read according to the data identifier of the data to be read;
- the return module 607 is configured to return the data to be read to the second terminal.
- the reading module 606 is further configured to determine whether the data identifier of the data to be read is included in the first storage record; when the first storage record includes the data identifier When the data identification of the data to be read is performed, the first storage path of the data to be read is determined from the first storage record according to the data identification of the data to be read, and according to the first A storage path to read the data to be read from the first solid-state hard disk; when the first storage record does not include the data identifier of the data to be read, according to the data of the data to be read Data identification, determining the second storage path of the data to be read from the second storage record, and reading the to-be-read from the cloud server according to the second storage path of the data to be read data.
- the reading module 606 is further configured to determine whether a data identifier of the data to be read is included in a third storage record, and the third storage record is a backup of the cache server The storage record of the server; when the third storage record contains the data identifier of the data to be read, determine the to-be-read from the third storage record according to the data identifier of the data to be read The third storage path of the data, according to the third storage path of the data to be read, read the data to be read from the second solid-state hard disk in the backup server; when the third storage record does not When the data identification of the data to be read is included, the second storage path of the data to be read is determined from the second storage record according to the data identification of the data to be read.
- a cache server is provided in the cloud storage system, and a solid state hard disk is used to store data in the cache server, and the characteristics of the solid state hard disk can ensure that data is written and refreshed to a new disk in real time. Therefore, when the first terminal is storing data, the cache server preferentially writes the data to be stored to the cache server, and then the cache server background is asynchronously transferred to the cloud server for persistent storage. Therefore, the cache server can quickly store and download data, which can support the user's demand for data to be stored and retrieved.
- the cloud storage-based data storage device provided in the above embodiment is based on the cloud storage data storage, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
- the cloud storage-based data storage device provided in the foregoing embodiment and the cloud storage-based data storage method embodiment belong to the same concept. For the specific implementation process, see the method embodiment, and details are not described here.
- FIG. 10 is a schematic structural diagram of a cache server provided by an embodiment of the present application.
- the cache server 1000 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units (CPU) 1001 And one or more memories 1002, wherein at least one instruction is stored in the memory 1002, and the at least one instruction is loaded and executed by the processor 1001 to implement the methods provided by the foregoing method embodiments.
- the server may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output.
- the server may also include other components for implementing device functions, which will not be repeated here.
- An embodiment of the present application provides a non-volatile computer-readable storage medium for storing a computer program, and the computer program is loaded by a processor to execute the instructions of the cloud storage-based data storage method.
- the program may be stored in a computer-readable storage medium.
- the mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (18)
- 一种基于云存储的数据存储方法,其特征在于,所述方法应用于缓存服务器,所述方法包括:接收第一终端发送的存储请求,所述存储请求携带待存储数据;将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
- 根据权利要求1所述的方法,其特征在于,所述定期将所述第一固态硬盘中待转存数据转存至云服务器中,包括:定期监测所述第一固态硬盘的空间利用率;当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
- 根据权利要求2所述的方法,其特征在于,所述从所述第一固态硬盘中选择所述待转存数据,包括:根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
- 根据权利要求1所述的方法,其特征在于,所述将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储 记录中之后,所述方法还包括:将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
- 根据权利要求4所述的方法,其特征在于,所述将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中之后,所述方法还包括:在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
- 根据权利要求1-5任一项权利要求所述的方法,其特征在于,所述方法还包括:接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;根据所述待读取数据的数据标识,读取所述待读取数据;向所述第二终端返回所述待读取数据。
- 根据权利要求6所述的方法,其特征在于,所述根据所述待读取数据的数据标识,读取所述待读取数据,包括:确定所述第一存储记录中是否包含所述待读取数据的数据标识;当所述第一存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
- 根据权利要求7所述的方法,其特征在于,所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径之前,所述方法还包括:确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;当所述第三存储记录中不包含所述待读取数据的数据标识时,执行所述根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径的步骤。
- 一种基于云存储的数据存储装置,其特征在于,所述装置应用于缓存服务器,所述装置包括:接收模块,用于接收第一终端发送的存储请求,所述存储请求携带待存储数据;存储模块,用于将所述待存储数据存储在所述缓存服务器中的第一固态硬盘中,将所述待存储数据的数据标识与其在所述第一固态硬盘中的第一存储路径的对应关系存储在第一存储记录中;转存模块,用于定期将所述第一固态硬盘中待转存数据转存至云服务器中,将所述待转存数据的数据标识与其在所述云服务器中的第二存储路径的对应关系存储在第二存储记录中。
- 根据权利要求9所述的装置,其特征在于,所述转存模块,还用于定期监测所述第一固态硬盘的空间利用率;当所述空间利用率大于指定利用率时,从所述第一固态硬盘中选择所述待转存数据,将所述待转存数据存储到所述云服务器中;记录所述待转存数据的转存时间,在当前时间与所述转存时间之间的差值超过第一指定时长时,将所述待转存数 据从所述第一固态硬盘中删除,将所述第一存储记录中所述待转存数据的数据标识与第一存储路径的对应关系删除。
- 根据权利要求10所述的装置,其特征在于,所述转存模块,还用于根据所述第一固态硬盘中数据的存储时间,从所述第一固态硬盘中选择存储时间离当前时间最远的数据作为待转存数据;或者,所述转存模块,还用于根据所述第一固态硬盘中数据的热度值,从所述第一固态硬盘中选择热度值最小的数据作为待转存数据,数据的热度值用于指示所述数据被读取的概率大小。
- 根据权利要求9所述的装置,其特征在于,所述装置还包括:备份模块,用于将所述待存储数据备份至备份服务器中的第二固态硬盘中,将所述待存储数据的数据标识与其在所述第二固态硬盘中的第三存储路径的对应关系存储在第三存储记录中。
- 根据权利要求12所述的装置,其特征在于,所述装置还包括:删除模块,用于在当前时间与所述待转存数据的转存时间之间的差值超过第二指定时长时,将所述待转存数据从所述第二固态硬盘中删除;将所述第三存储记录中的所述待转存数据的数据标识与第三存储路径的对应关系删除。
- 根据权利要求9-13任一项权利要求所述的装置,其特征在于,所述装置还包括:所述接收模块,还用于接收第二终端发送的读取请求,所述读取请求携带待读取数据的数据标识;读取模块,用于根据所述待读取数据的数据标识,读取所述待读取数据;返回模块,用于向所述第二终端返回所述待读取数据。
- 根据权利要求14所述的装置,其特征在于,所述读取模块,还用于确定所述第一存储记录中是否包含所述待读取数据的数据标识;当所述第一存储记录中包含所述待读取数据的数据标识时,根据 所述待读取数据的数据标识,从所述第一存储记录中确定所述待读取数据的第一存储路径,根据所述待读取数据的第一存储路径,从所述第一固态硬盘中读取所述待读取数据;当所述第一存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径,根据所述待读取数据的第二存储路径,从所述云服务器中读取所述待读取数据。
- 根据权利要求15所述的装置,其特征在于,所述读取模块,还用于确定第三存储记录中是否包含所述待读取数据的数据标识,所述第三存储记录为所述缓存服务器的备份服务器的存储记录;当所述第三存储记录中包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第三存储记录中确定所述待读取数据的第三存储路径,根据所述待读取数据的第三存储路径,从所述备份服务器中的第二固态硬盘中读取所述待读取数据;当所述第三存储记录中不包含所述待读取数据的数据标识时,根据所述待读取数据的数据标识,从所述第二存储记录中确定所述待读取数据的第二存储路径。
- 一种基于云存储的数据存储装置,其特征在于,所述装置包括至少一个处理器;和至少一个存储器;所述至少一个存储器存储有一个或多个程序,所述一个或多个程序被配置成由所述至少一个处理器执行,所述一个或多个程序包含用于进行如权利要求1至8任一项权利要求所述的基于云存储的数据存储方法的指令。
- 一种非易失性计算机可读存储介质,其特征在于,用于存储计算机程序,所述计算机程序通过处理器进行加载来执行如权利要求1至8任一项权利要求所述的基于云存储的数据存储方法的指令。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811348900.XA CN111176549B (zh) | 2018-11-13 | 2018-11-13 | 基于云存储的数据存储方法、装置和存储介质 |
CN201811348900.X | 2018-11-13 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020098654A1 true WO2020098654A1 (zh) | 2020-05-22 |
Family
ID=70657337
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/117570 WO2020098654A1 (zh) | 2018-11-13 | 2019-11-12 | 基于云存储的数据存储方法、装置和存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN111176549B (zh) |
WO (1) | WO2020098654A1 (zh) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112596682A (zh) * | 2020-12-28 | 2021-04-02 | 郝东东 | 一种区块链用存储装置及存储方法 |
CN114461146A (zh) * | 2022-01-26 | 2022-05-10 | 北京百度网讯科技有限公司 | 云存储的数据处理方法、装置、系统、设备、介质及产品 |
CN115202312A (zh) * | 2022-09-16 | 2022-10-18 | 山东神驰化工集团有限公司 | 一种智能式化工安全生产巡控系统 |
CN115543185A (zh) * | 2022-03-30 | 2022-12-30 | 中国移动通信集团四川有限公司 | 数据存储管理方法及系统、装置、计算机可读存储介质 |
WO2023060943A1 (zh) * | 2021-10-14 | 2023-04-20 | 华为技术有限公司 | 一种流控方法及装置 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115065694B (zh) * | 2022-07-29 | 2024-02-09 | 苏州浪潮智能科技有限公司 | 一种云存储数据中转上传系统、方法、设备及介质 |
CN116340055B (zh) * | 2023-03-28 | 2024-03-12 | 上海威固信息技术股份有限公司 | 一种固态硬盘的数据恢复方法及系统 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060004957A1 (en) * | 2002-09-16 | 2006-01-05 | Hand Leroy C Iii | Storage system architectures and multiple caching arrangements |
CN103034592A (zh) * | 2012-12-05 | 2013-04-10 | 华为技术有限公司 | 数据处理方法和装置 |
CN103795756A (zh) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | 分层存储方法和系统、对象访问控制器 |
CN104618482A (zh) * | 2015-02-02 | 2015-05-13 | 浙江宇视科技有限公司 | 访问云数据的方法、服务器、传统存储设备、架构 |
CN105491149A (zh) * | 2015-12-26 | 2016-04-13 | 深圳市金立通信设备有限公司 | 一种数据存储方法及终端 |
CN106911776A (zh) * | 2017-02-24 | 2017-06-30 | 郑州云海信息技术有限公司 | 一种云存储设备的管理方法及装置 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103873501B (zh) * | 2012-12-12 | 2017-07-18 | 华中科技大学 | 一种云备份系统及其数据备份方法 |
CN103605585B (zh) * | 2013-11-06 | 2017-01-18 | 上海爱数信息技术股份有限公司 | 一种基于数据发现的智能备份方法 |
CN106161523B (zh) * | 2015-04-02 | 2019-11-22 | 腾讯科技(深圳)有限公司 | 一种数据处理方法和设备 |
CN107786599B (zh) * | 2016-08-26 | 2020-07-17 | 北京计算机技术及应用研究所 | 内存云系统 |
CN108093207B (zh) * | 2016-11-23 | 2021-03-16 | 杭州海康威视数字技术股份有限公司 | 一种数据存储、获取方法及装置 |
-
2018
- 2018-11-13 CN CN201811348900.XA patent/CN111176549B/zh active Active
-
2019
- 2019-11-12 WO PCT/CN2019/117570 patent/WO2020098654A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060004957A1 (en) * | 2002-09-16 | 2006-01-05 | Hand Leroy C Iii | Storage system architectures and multiple caching arrangements |
CN103795756A (zh) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | 分层存储方法和系统、对象访问控制器 |
CN103034592A (zh) * | 2012-12-05 | 2013-04-10 | 华为技术有限公司 | 数据处理方法和装置 |
CN104618482A (zh) * | 2015-02-02 | 2015-05-13 | 浙江宇视科技有限公司 | 访问云数据的方法、服务器、传统存储设备、架构 |
CN105491149A (zh) * | 2015-12-26 | 2016-04-13 | 深圳市金立通信设备有限公司 | 一种数据存储方法及终端 |
CN106911776A (zh) * | 2017-02-24 | 2017-06-30 | 郑州云海信息技术有限公司 | 一种云存储设备的管理方法及装置 |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112596682A (zh) * | 2020-12-28 | 2021-04-02 | 郝东东 | 一种区块链用存储装置及存储方法 |
WO2023060943A1 (zh) * | 2021-10-14 | 2023-04-20 | 华为技术有限公司 | 一种流控方法及装置 |
CN114461146A (zh) * | 2022-01-26 | 2022-05-10 | 北京百度网讯科技有限公司 | 云存储的数据处理方法、装置、系统、设备、介质及产品 |
CN114461146B (zh) * | 2022-01-26 | 2024-05-07 | 北京百度网讯科技有限公司 | 云存储的数据处理方法、装置、系统、设备、介质及产品 |
CN115543185A (zh) * | 2022-03-30 | 2022-12-30 | 中国移动通信集团四川有限公司 | 数据存储管理方法及系统、装置、计算机可读存储介质 |
CN115543185B (zh) * | 2022-03-30 | 2023-12-19 | 中国移动通信集团四川有限公司 | 数据存储管理方法及系统、装置、计算机可读存储介质 |
CN115202312A (zh) * | 2022-09-16 | 2022-10-18 | 山东神驰化工集团有限公司 | 一种智能式化工安全生产巡控系统 |
CN115202312B (zh) * | 2022-09-16 | 2022-12-27 | 山东神驰化工集团有限公司 | 一种智能式化工安全生产巡控系统 |
Also Published As
Publication number | Publication date |
---|---|
CN111176549A (zh) | 2020-05-19 |
CN111176549B (zh) | 2021-05-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020098654A1 (zh) | 基于云存储的数据存储方法、装置和存储介质 | |
US9430321B2 (en) | Reconstructing data stored across archival data storage devices | |
US9436571B2 (en) | Estimating data storage device lifespan | |
US20220035555A1 (en) | Index Lifecycle Management | |
US8615485B2 (en) | Method and system for managing weakly mutable data in a distributed storage system | |
US9189493B2 (en) | Object file system | |
US20150356125A1 (en) | Method for data placement based on a file level operation | |
US20150254320A1 (en) | Using colocation hints to facilitate accessing a distributed data storage system | |
WO2019001521A1 (zh) | 数据存储方法、存储设备、客户端及系统 | |
US9557938B2 (en) | Data retrieval based on storage device activation schedules | |
CN103917962A (zh) | 读取存储在存储系统上的文件 | |
US20140281301A1 (en) | Elastic hierarchical data storage backend | |
US9436524B2 (en) | Managing archival storage | |
US20130325932A1 (en) | Electronic device and method for storing distributed documents | |
CN115202929B (zh) | 数据库集群备份系统 | |
JP2015527620A (ja) | 計算機システム、サーバ、及び、データ管理方法 | |
CN110502187B (zh) | 一种快照回滚方法及装置 | |
WO2014153931A1 (zh) | 文件存储方法、装置、访问客户端及元数据服务器系统 | |
US9430149B2 (en) | Pipeline planning for low latency storage system | |
US11157511B2 (en) | Physical replication of database | |
CN109144403B (zh) | 一种用于云盘模式切换的方法与设备 | |
US9430152B2 (en) | Data device grouping across data storage device enclosures for synchronized data maintenance | |
US9766677B2 (en) | Cascading startup power draws of enclosures across a network | |
CN112286448B (zh) | 对象访问方法、装置、电子设备及机器可读存储介质 | |
WO2018077092A1 (zh) | 应用于分布式文件系统的存盘方法、装置及分布式文件系统 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19883815 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19883815 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19883815 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 04/02/2022) |