RU2435236C1 - System and method of recording data into cloud storage - Google Patents

Abstract

FIELD: information technology. ^ SUBSTANCE: system for recording data into cloud storage, having cloud storage and at least one user device containing a driver for the cloud storage, which includes a router, a prediction module, cache, cache manager, a connector to the cloud storage, a module for routing system events and user application query history, wherein the cache and the query history are contained a local file system, wherein the user device is configured to run the user application in the operating system; the router is configured to intercept user application queries to the file system and, if the query is directed the cloud storage, redirect the query to the cache manager and the prediction module, and also configured to return data obtained from the cache manager to the user application. ^ EFFECT: high efficiency of operation owing to more accurate prediction by using events occurring in the operating system to predict. ^ 3 cl, 4 dwg

Description

The invention relates to the field of data distribution in memory systems or architectures, and in particular to systems and methods for writing data to a cloud storage and can be applied to local devices (for example, PCs or mobile devices) with a limited amount of resources connected to an external cloud data storage (cloud storage).

Recently, cloud computing or cloud computing technologies have been increasingly used, which allow the use of external computing resources (processor time, memory, networks) to expand the capabilities of local devices (PCs, mobile devices). The methods of using external storage devices (external memory) have already found wide application in practice. So, dropbox.com service and its analogues allow you to rent storage devices and use them from PC or mobile devices. This allows users to access the same data from any PC or device connected to the Internet. At the same time, data stored on a PC or mobile device is fully synchronized with external cloud storage.

Mobile devices can use cloud storage to expand storage capacity. Having 1-10 GB of memory in a mobile device, you can use an additional 100 GB from the "cloud" without increasing the physical size of the device’s memory and, accordingly, its cost. But these services use full data synchronization, which is not applicable in mobile devices with a small amount of memory.

For the efficient use of cloud disks in mobile phones, it is necessary to perform efficient data caching, i.e. you need to choose which of the data should be on the device and which to remain in the "cloud". The system module that performs this function is called the cache manager. It can be located both on a mobile device and in the cloud. In various technical solutions, the cache manager uses various data to determine the cached subset of the data, in particular:

cache all data (synchronization);

cached data is set by the user;

predict cached data based on statistics of requests for data from the "cloud".

A well-known client is the Dropbox cloud storage service (http://dropbox.com) [1], which allows users to store their data in external storage. A client program is installed on the user's device. The data is stored on the server, the client synchronizes them with the user device.

This system does not use a cache of a small, predetermined size, and forecasting algorithms based on statistics of user requests to the “cloud” and system events are not used to select a cached subset of data, which reduces the efficiency of the system. In addition, a serious drawback of this solution is that the system can work only with one cloud storage (dropbox), which makes the user dependent on the service provider, impairing the usability of the system.

US Pat. No. 6,032,227 [2] describes a system for delivering user data from storage to the memory of a user device having a limited cache. The system consists of the following elements: file system driver (IFS), cache manager, network file system on the server (NFS). The file system driver intercepts system calls to the file system from various user applications. The driver then redirects the received system call to the cache manager, which, analyzing the system call, determines where the requested file is located and addresses either the device’s local file system or the network file system (NFS) on the server. The cache manager keeps a history of recently used files (statistics of user requests). In the case when there is not enough space in the cache for the newly needed file, the “cache manager”, based on the history of recently used files, deletes the oldest files from the cache until the necessary disk space is freed up.

The disadvantages of this system are as follows. In some cases (for example, when the cache overflows), user files are downloaded from the storage to the device’s cache at the user's request, so the user does not work with the storage transparently, but in a dialogue mode. At the same time, the forecast of user needs is not built, as a result of which, for each operation of reading previously unused data, there is a delay for a time until this data is loaded into the cache. Only one network file system (NIS) is used to transfer data from the server, which reduces the versatility of the system. Elements of the system, including the IFS mechanism and the cache manager, operate at the kernel level of the OS, which imposes significant restrictions on the amount of memory used and makes it impossible to use effective algorithms for predicting user data needs.

There is a system for delivering content (structured data) from a personal computer to a mobile device via a network (US Patent Application Laid-Open No. 2010/0174735) [3]. In this system, a personal computer is a server on which various content is stored, data about users and the history of their requests, as well as forecast, content conversion, and quality of service (QoS) modules. A user on a mobile device through specialized software generates a search query for specific content and sends it to the server. The server, according to this request, makes a selection of suitable content and sends it to the user, then, based on this and previous requests of the user, he makes an assumption about what other content may interest the user, and sends it too. Content can be delivered to the device in both its original and converted form. In the latter case, before sending the content to the user, the content passes through the content conversion module and is converted to a format suitable for the mobile device (for example, from avi to 3gp).

The disadvantages of this solution are as follows. A data warehouse is a computer system capable of performing some calculation. The forecast module is located on this computing system, which does not interfere with the use of this solution for working with cloud storage services that are not able to perform calculations. In addition, the forecast of the data necessary for the user is based on a search query generated by the user, which implies working with the user in a dialogue mode.

Closest to the claimed invention are a system and method for delivering data from an arbitrary network resource (www, ftp ...) to a mobile device by means of the network described in US Patent Application Laid-Open No. 2005/0108322 [4]. As a data transfer intermediary, some server is used, access to which is carried out wirelessly from a mobile device. The server is a “computing environment” (PC, cluster, distributed system), which contains three software modules: a storage module, a prediction module, and a delivery module. Specialized software is installed on the mobile device, which allows you to make requests to various network resources through an intermediate server and receive requested data from it. When the user makes a request, this request is sent to the intermediate server over the wireless network, where the request is stored in the storage module. Based on the information contained in the storage module, the forecast module predicts a subsequent user request, downloads the predicted information, stores it in the storage module, and transfers it to the delivery module. The availability of predicted information in the storage module provides, with subsequent access to it, quick access to data over a wireless local area network without Internet access. Prediction of subsequent user activity is determined using Markov chains. The transit delivery module transmits to the mobile device the requested resource and resources that the user will presumably need in the next step (the result of the forecast module). These system and method are selected as a prototype of the claimed invention.

The disadvantages of the system and method of the prototype are as follows. The prototype uses an external computing system, access to which is via a wireless local area network, which reduces the versatility of using the system outside of local networks with a deployed auxiliary server.

The objective of the invention is the creation of a system and method of recording data in a cloud storage with enhanced usability: to ensure transparent operation of the user with large amounts of data with a small cache due to the lack of dialogue between the system and the user in the dialogue mode, as well as providing the possibility of the user choosing data warehousing regardless of the storage service provider; and with improved work efficiency, by constructing a more accurate forecast by using not only statistics for user queries to forecast, but also events occurring in the operating system.

The problem is solved by creating an improved system for writing data to the cloud storage, while the inventive system contains: cloud storage and at least one user device containing a cloud storage driver, said driver including a router, prediction module, cache, manager a cache, a cloud storage connector, a system events tracing module and a request history of a user application, and the cache and request history are contained in the local file system e, while the user device is configured to execute the user application in the operating system; the router is capable of intercepting requests of the user application to the file system, and if the request is directed to the cloud storage, then redirecting the request to the cache manager and forecasting module, as well as with the ability to return data received from the cache manager to the user application; the cache manager is configured to determine whether the requested resource is in the cache or not, and if the requested data is not in the cache, then send a request to download data to the connector to the cloud storage, and also with the ability to return data to the router after downloading; moreover, when a request arrives at the router, the router is also configured to send request data to the forecasting module, which is configured to form an assumption about which files the user will need in the next step, and send an application for downloading them to the cache manager based on the current request and user activity history, and if this data is not in the cache, a cache manager is used that is capable of downloading this data from the cloud storage using the connector to the cloud at the store.

In this context, the term “connector” is understood in the sense as defined by the recommendations for Microsoft Exchange Server (see http://en.wikipedia.org/wiki/%D0%9A%D0%BE%D0%BD%D0% BD% D0% B5% D0% BA% D1% 82% D0% BE% D1% 80_% 28Exchange% 29) [5], namely, as a configured connection between mail servers in different routing groups or mail systems. In this case, specific connectors can be made in the form of connectors adapted to connections with specific types of servers.

For the functioning of the system, it is important that the user device is selected from a set of user devices containing a mobile device and a personal computer.

The problem is also solved by creating a way to write data to the cloud storage in a system that contains cloud storage and at least one user device that contains a cloud storage driver, which includes a router, forecast module, cache, cache manager, connector to cloud storage, the module for tracing system events and the request history of the user application, while the method involves the following operations:

- form in the user device the requests of the user application to the file system;

- intercept user application requests to the file system through a router;

- if the request is directed to the cloud storage, then the request is sent to the cache manager and forecasting module through the router;

- determine with the help of the cache manager whether the requested resource is in the cache or not;

- if the resource is in the cache, then with the help of the cache manager send a request for the user application to the cached file;

- if the requested data is not in the cache, then with the help of the cache manager they send a request to download data to the cloud storage connector;

- after downloading the data, the data is returned using the cache manager to the router, by which the data is returned to the user application;

- moreover, when a request arrives at the router, the request data is sent to the forecasting module via the router, make an assumption about what files the user will need in the next step, and send a request for their download to the cache manager, based on the current request and the history of user activity through the module forecasting, and if this data is not in the cache, download it using the cache manager from the cloud storage using the connector to the cloud storage.

For a better understanding of the claimed invention the following is a detailed description with the involvement of the relevant graphic materials.

Figure 1. A general diagram of a cloud storage data recording system in accordance with the invention.

Items:

101 - user;

110 is a mobile device;

111 - cloud storage driver;

113.1 - cache on a mobile device;

113.2- cache on a personal computer;

120 - personal computer;

130 - network;

140 - cloud storage;

145 - data.

Figure 2. Cloud driver circuitry made according to the invention.

Items:

112 - router;

113 - cache;

114 - module tracing system events;

115 - cache manager;

116 - history;

117 - forecasting module;

118 - connector to the cloud storage;

119.1, 119.2 - adapted connectors to cloud storage.

Figure 3. A flowchart of a method for recording data to a cloud storage according to the invention.

Figure 4. A detailed diagram of a cloud storage system according to the invention.

Items:

420 - user applications;

450 - local FS;

500.1, 500.2 - cloud storage.

The claimed system for recording data to the cloud storage can be installed on a mobile device. The user installs the system in a device to expand the used disk space without installing additional physical memory modules (flash cards). The user selects one of the service providers (supported by the cloud storage system) and, if necessary, enters his authorization data (username and password for the store). As a parameter of the system, the maximum cache size is set.

When connecting to cloud storage, the connector module determines which storage should be used. The selection is based on the service provider selected by the user.

During operation, the system intercepts requests from user applications to the file system via a router and passes them to the cache manager and forecasting module. Emerging system events (starting a process, detecting a new network, connecting to a device, changing GPS coordinates) are recorded in the system events tracing module and passed to the forecasting module.

The cache manager analyzes the intercepted request and determines whether the requested file is in the cache or not. If the file is already in the cache, the system uses the acknowledged file. Otherwise, the system downloads the required file from the cloud storage. The cache manager stores cache data in the local file system, while ensuring that the size of the cache data does not exceed the specified maximum cache size.

During the operation of the system, the forecasting module collects the history of accesses to the file system and system events in the form of history, which, like cache data, is stored on the local file system. History is used by the module to predict future user needs. If the forecasting module calculates that the probability of a user requesting a specific file is high, then it generates a request to the cache manager module to load this file.

All work on downloading and deleting files from the cache is transparent to the user.

Consider the operation of the claimed system and method of recording data in a cloud storage in the case of two local devices: a mobile device and a personal computer (Figure 1-4). When user application 420 sends a request to access file system 450, it is intercepted by router 112. Router 112 redirects the request to cache manager 115 and prediction module 117.

The cache manager 115 determines where the requested resource is actually located: in cache 113 or not. If the requested data is in cache 113, then this cached file is accessed. If the requested data is not in the cache 113, the cache manager 115 determines if there is enough space in the cache 113 to load the requested file and free up space if necessary. After that, the cache manager 115 sends a request to download data to the cloud storage connector 118.

The cloud storage connector 118, depending on the cloud storage 500 used (500.1 or 500.2), selects the appropriate adapted connector 119 (119.1 or 119.2) and downloads the requested data to the cache 113. After downloading the data, the cache manager 115 returns data to the router 112, and that returns data user application 420.

Also, when a request arrives at router 112, it sends the request data to prediction module 117. Prediction module 117, based on the current request and user activity history 116, makes an assumption about which files the user will need in the next step, and sends a request to download them to cache manager 115, and if this data is not in cache 113, then manager 115 cache downloads them from cloud storage 500 (500.1 or 500.2) using connector 118 to the cloud storage.

Prediction module 117 obtains system event information from system event tracing module 114. Prediction module 117, based on the current system event and user activity history 116, makes an assumption about which files the user will need in the next step, and sends a request to download them to cache manager 115, and if this data is not in cache 113, then the manager 115 caches downloads them from cloud storage 500 (500.1 or 500.2) using connector 118 to the cloud storage.

The claimed invention describes a system for caching data on a mobile device, taking into account the following features specific to cloud storage and mobile devices:

- Cloud storage cannot contain any computing elements (including caching module). Storage performs a strictly defined data storage function.

- The system must be operational with various cloud storage services (dropbox, files.ovi.com, etc.).

- The use of storage should be transparent to the user. The user cannot set the data to cache.

- The cache manager should make a forecast based not only on statistics of data requests, but also on the basis of system events of a mobile device (starting a process, detecting a new network, connecting to a device, changing GPS coordinates).

Although the above embodiment of the invention has been set forth to illustrate the present invention, it is clear to those skilled in the art that various modifications, additions and substitutions are possible without departing from the scope and meaning of the present invention disclosed in the attached claims.

Claims (3)

1. A system for writing data to a cloud storage containing cloud storage and at least one user device that contains a cloud storage driver, which includes a router, a prediction module, a cache, a cache manager, a cloud storage connector, a system trace module events and request history of the user application, and the cache and history of requests are contained in the local file system, while the user device is configured to execute the user application living in the operating system; the router is capable of intercepting requests from the user application to the file system and, if the request is directed to the cloud storage, redirecting the request to the cache manager and forecasting module, and also with the ability to return data received from the cache manager to the user application; the cache manager is configured to determine whether the requested resource is in the cache or not, and if the requested data is not in the cache, send a request to download data to the cloud storage connector, as well as to return data to the router after downloading; moreover, the router is also configured to send, upon request to the router, the request data to the forecasting module, which is configured to form an assumption about which files the user will need in the next step and send an application for their download to the cache manager, based on the current request and history of user activity, moreover, the cache manager is configured to download this data from the cloud storage using the connector to the cloud storage if such data is available They are in cache. 2. The system according to claim 1, characterized in that the user device is selected from a set of user devices, including a mobile device and a personal computer. 3. A method of writing data to the cloud storage in a system that contains cloud storage and at least one user device that contains a cloud storage driver, which includes a router, a prediction module, a cache, a cache manager, a cloud storage connector, a module for tracing system events and a request history of a user application, while the method provides for the following operations:
form in the user device the requests of the user application to the file system;
intercept user application requests to the file system using a router;
if the request is directed to the cloud storage, send a request to the cache manager and forecast module using a router;
determine with the help of the cache manager whether the requested resource is in the cache or not;
if the resource is in the cache, then, with the help of the cache manager, a user application request is sent to the cached file;
if the requested data is not in the cache, then they send a request using the cache manager to download data to the cloud storage connector;
after downloading the data, the data is returned using the cache manager to the router, through which the data is returned to the user application;
moreover, when a request arrives at the router, the request data is sent to the forecasting module using the router, make an assumption about which files the user will need in the next step, and send a request for their download to the cache manager based on the current request and the history of user activity performed using the forecasting module, and if this data is not in the cache, download it using the cache manager from the cloud storage using the connector to the cloud storage.

Images