CN114691033A

CN114691033A - Data copying method, data storage system control method, device, equipment and medium

Info

Publication number: CN114691033A
Application number: CN202210147250.2A
Authority: CN
Inventors: 陈爽; 仓利基; 肖国平
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-02-17
Filing date: 2022-02-17
Publication date: 2022-07-01

Abstract

The embodiment of the disclosure discloses a data replication method, a data storage system control method, a data replication device, a data storage system control device and a data storage system control medium, wherein the data replication method comprises the steps of obtaining a data capture bandwidth and a data access frequency, the data capture bandwidth is a bandwidth occupied when data are captured from a source data storage space to a target data storage space, and the data access frequency is the number of times of capturing the data from the source data storage space in unit time; in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the target access frequency, data in the source data storage space is copied to the target data storage space. The scheme can enable a user to directly read the data originally stored in the source data storage space from the target data storage space without grabbing the data from the source data storage space to the target data storage space, thereby saving the communication cost, reducing the data processing burden, simultaneously reducing the time delay when the user reads the data, and improving the user experience.

Description

Data copying method, data storage system control method, device, equipment and medium

Technical Field

The present disclosure relates to the field of network technologies, and in particular, to a data replication method, a data storage system control method, an apparatus, a device, and a medium.

Background

In recent years, with the development of data processing technology, multiple parties not in the same place can jointly complete data processing, and accordingly, since a local data storage system cannot meet the requirement of reading multiple data in the above scenario, in the related art, data can be stored in a data storage system on line, a user with a data reading requirement can be connected to the data storage system to access the data storage system on line through any internet-connectable terminal device at any time and any place, so that data stored in the data storage system can be read more conveniently. In order to reduce the time delay of reading data by a user, a plurality of data centers located in different regions may be set, and the corresponding data centers bear corresponding storage spaces, so that the user may read data from the storage space borne by the data center located in the same region as the user.

In the related art, although the time delay for the user to read data from the storage space carried by the data center in the same region is low, when the data requested to be read by the user is not stored in the storage space, the online data storage system may capture the data requested to be read by the user from the source storage space according to the configured source storage space address, and store the data requested to be read by the user into the storage space, so that the data can be read by the user.

In the above scheme, although it can be ensured that a user can read data requested to be read by the user from a storage space carried by a data center located in the same region as the user, when communication resources between the storage space and a source storage space corresponding to the storage space are scarce, the source storage space is accessed to capture data with a large capacity and store the data into the storage space in a short time, so that more communication resources are occupied, which results in a high captured data, or a communication service provider may forcibly limit the occupation of the data storage system on the communication resources; in addition, when the data processing resources of the data storage system are scarce, the data are frequently fetched from the source storage space and stored in the storage space, which may increase the data processing burden of the data storage system, and may cause the data storage system to limit the number of data fetches per unit time, thereby impairing the user experience.

Disclosure of Invention

In order to solve the problems in the related art, embodiments of the present disclosure provide a data replication method, a data storage system control method, an apparatus, a device, and a medium.

In a first aspect, an embodiment of the present disclosure provides a data replication method, where the method is applied to a data storage system, and the method includes:

acquiring data capture bandwidth and data access frequency, wherein the data capture bandwidth is the bandwidth occupied when data are captured from a source data storage space to a target data storage space, and the data access frequency is the number of times of capturing the data from the source data storage space in unit time;

in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the target access frequency, data in the source data storage space is copied to the target data storage space.

With reference to the first aspect, in a first implementation manner of the first aspect, in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the access frequency threshold, before copying the data in the source data storage space to the target data storage space, the method further includes:

acquiring data replication configuration information, wherein the data replication configuration information comprises a first data storage space mark, a second data storage space mark and threshold configuration information, and the threshold configuration information comprises at least one of a configuration single block and configuration access frequency;

determining the configuration bandwidth as a target bandwidth in response to the first data storage space indicating that the indicated data storage space is a source data storage space and the second data storage space indicating that the indicated data storage space is a target data storage space;

and/or determining the configuration access frequency as the target access frequency in response to the first data storage space indicating that the indicated data storage space is the source data storage space and the second data storage space indicating that the indicated data storage space is the target data storage space.

With reference to the first aspect or the first implementation manner of the first aspect, in a second implementation manner of the first aspect, before the data in the source data storage space is copied to the target data storage space, the method further includes:

acquiring data capture history of data captured from a source data storage space;

determining target data which are not captured in a source data storage space according to the data capturing history;

copying data in the source data storage space to the target data storage space, comprising:

the target data is copied from the source data storage space to the target data storage space.

With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the data fetch history includes an accessed time of a data directory in which data is located in the source data storage space;

determining uncaptured target data in a source data storage space according to the data capture history, comprising:

and determining a target data directory of which the last time the source data storage space is accessed is earlier than the target access time, and determining the data in the target data directory as the target data.

With reference to the first aspect or the first implementation manner of the first aspect, in a fourth implementation manner of the first aspect, before the data in the source data storage space is copied to the target data storage space, the method further includes:

acquiring the capacity of data files in a source data storage space, and sequencing the data files in the source data storage space according to the capacity in a sequence from small to large;

copying data in the source data storage space to the target data storage space, including:

and copying the sequenced data files to a target data storage space in sequence.

In a second aspect, an embodiment of the present disclosure provides a method for controlling a data storage system, where the method is applied to a control end of the data storage system, and the method includes:

acquiring data replication configuration information, wherein the data replication configuration information comprises a first data storage space mark, a second data storage space mark and threshold configuration information, and the threshold configuration information comprises at least one of configuration bandwidth and configuration access frequency;

and sending the data replication configuration information.

With reference to the second aspect, in a first implementation manner of the second aspect, the data replication configuration information further includes at least one of first data replication indication information and second data replication indication information;

the first data copying indication information is used for indicating to acquire data capturing history of data captured from the source data storage space and determining target data which are not captured in the source data storage space according to the data capturing history;

and the second data replication indication information is used for indicating the capacity of the data files in the source data storage space, and sequencing the data files in the source data storage space according to the capacity in the order from small to large.

In a third aspect, an embodiment of the present disclosure provides a data copying apparatus, where the apparatus includes:

the bandwidth frequency acquisition module is configured to acquire a data capture bandwidth and a data access frequency, wherein the data capture bandwidth is a bandwidth occupied by a target data storage space when capturing data from a source data storage space, and the data access frequency is the number of times of capturing data from the source data storage space in unit time;

the data copying module is configured to respond to the condition that the data capture bandwidth is larger than or equal to the target bandwidth and/or the data access frequency is larger than or equal to the target access frequency, and copy the data in the source data storage space to the target data storage space.

In a fourth aspect, an embodiment of the present disclosure provides a data storage system control apparatus, where the apparatus includes:

the data replication management system comprises a configuration information acquisition module, a data replication management module and a data replication management module, wherein the configuration information acquisition module is configured to acquire data replication configuration information, the data replication configuration information comprises a first data storage space mark, a second data storage space mark and threshold configuration information, and the threshold configuration information comprises at least one of configuration bandwidth and configuration access frequency;

a configuration information transmitting module configured to transmit data duplication configuration information.

In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including a memory and at least one processor; the memory is configured to store one or more computer instructions that are executable by the at least one processor to implement the method steps of any one of the first aspect, the first implementation of the first aspect to the fourth implementation, the second aspect, and the first implementation of the second aspect.

In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having stored thereon computer instructions, which, when executed by a processor, implement the method steps of any one of the first aspect, the first implementation manner to the fourth implementation manner of the first aspect, the second aspect, and the first implementation manner of the second aspect.

In a seventh aspect, an embodiment of the present disclosure provides a computer program product, which includes computer instructions that, when executed by a processor, implement the method steps as in any one of the first aspect, the first implementation manner to the fourth implementation manner of the first aspect, the second aspect, and the first implementation manner of the second aspect.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

according to the technical scheme provided by the embodiment of the disclosure, data in the source data storage space is copied to the target data storage space by acquiring the data capture bandwidth and the data access frequency and responding to the situation that the data capture bandwidth is greater than or equal to the target bandwidth and/or the data access frequency is greater than or equal to the target access frequency, wherein the data capture bandwidth is a bandwidth occupied when capturing data from the source data storage space to the target data storage space, and the data access frequency is the number of times of capturing data from the source data storage space in a unit time, so that the data capture bandwidth and the data access frequency can reflect the amount of communication resources occupied when capturing data from the source data storage space to the target data storage space, and when the data capture bandwidth is greater than or equal to the target bandwidth, it can be considered that if the communication resources occupied by the data storage system are continuously increased, this may result in higher communication costs or may also result in the communication service provider forcing a limitation of the communication resources that can be used by the data storage system; when the data access frequency is greater than or equal to the target access frequency, it may be considered that if the data access frequency continues to increase, the data processing resources of the data storage system are occupied too much, which may result in an excessive data processing burden on the data storage system, and the data storage system may limit the number of data capturing times in unit time. In order to avoid the situation that the communication cost is too high, a communication service provider forcibly limits communication resources which can be used by the data storage system and the data capturing frequency of the data storage system in unit time, the data in the source data storage space is copied to the target data storage space, so that a user can directly read the data originally stored in the source data storage space from the target data storage space without capturing the data from the source data storage space to the target data storage space, the communication cost is saved, the data processing burden of the data storage system is reduced, meanwhile, the time delay when the user reads the data is reduced, and the user experience is improved.

According to the technical solution provided by the embodiment of the present disclosure, by obtaining the data replication configuration information, when the first data storage space in the data replication configuration information indicates that the indicated data storage space is the source data storage space and the indicated data storage space in the data replication configuration information is the target data storage space, it may be considered that the data replication configuration information is used for configuring the source data storage space and the target bandwidth and/or the target access frequency corresponding to the target data storage space, so that in response to the first data storage space indicating that the indicated data storage space is the source data storage space and the second data storage space indicating that the indicated data storage space is the target data storage space, the configuration bandwidth is determined as the target bandwidth and/or in response to the first data storage space indicating that the indicated data storage space is the source data storage space, and the second data storage space marks that the indicated data storage space is the target data storage space, and the configuration access frequency is determined as the target access frequency, so that the user can configure the target bandwidth and/or the target access frequency conveniently, and the user experience is improved.

According to the technical scheme provided by the embodiment of the disclosure, the data capture history of the data captured from the source data storage space is obtained, the target data which is not captured in the source data storage space is determined according to the data capture history, and the target data is copied from the source data storage space to the target data storage space, so that the data which is captured from the source data storage space can not be repeatedly copied, and the copying efficiency is improved.

According to the technical scheme provided by the embodiment of the disclosure, the accuracy of the target data which is not captured in the determined source data storage space can be improved by determining the target data directory of which the latest access time of the source data storage space is earlier than the target access time and determining the data in the target data directory as the target data.

According to the technical scheme provided by the embodiment of the disclosure, the data files with smaller capacity in the source data storage space can be preferentially copied to the target data storage space by acquiring the capacity of the data files in the source data storage space, sequencing the data files in the source data storage space according to the capacity in the order from small to large, and copying the sequenced data files to the target data storage space in sequence. When a user reads a data file, even if different data files have differences in capacity, the required reading times are all one time, and the data file with smaller capacity in the source data storage space is preferentially copied to the target data storage space, so that the times of capturing data from the source data storage space can be reduced as much as possible on the premise that the copying speed of the data file is not changed, and the burden of a data storage system is reduced.

According to the technical scheme provided by the embodiment of the disclosure, the data storage system can receive the data copying configuration information by acquiring the data copying configuration information and sending the data copying configuration information. When the first data storage space in the data replication configuration information indicates that the indicated data storage space is the source data storage space and the indicated data storage space in the data replication configuration information is the target data storage space, the data replication configuration information may be considered to be used for configuring the source data storage space and the target bandwidth and/or the target access frequency corresponding to the target data storage space, so that the data storage system may determine the configured bandwidth as the target bandwidth in response to the first data storage space indicating that the indicated data storage space is the source data storage space and the second data storage space indicating that the indicated data storage space is the target data storage space, and/or the data storage system may determine the data storage space indicated by the first data storage space as the source data storage space, and the second data storage space marks that the indicated data storage space is the target data storage space, and the configuration access frequency is determined as the target access frequency, so that the user can configure the target bandwidth and/or the target access frequency conveniently, and the user experience is improved.

According to the technical scheme provided by the embodiment of the disclosure, the data replication configuration information further comprises the first data replication indication information, so that the data storage system can respond to the first data replication indication information, acquire the data capture history of data captured from the source data storage space, and determine target data which is not captured in the source data storage space according to the data capture history, so that the target data can be conveniently replicated from the source data storage space to the target data storage space, the data captured from the source data storage space can be ensured not to be replicated repeatedly, and the replication efficiency is improved; by enabling the data replication configuration information to further include the second data replication indication information, the data storage system can obtain the capacity of the data files in the source data storage space in response to the second data replication indication information, and sequence the data files in the source data storage space according to the capacity in a sequence from small to large so as to sequentially replicate the sequenced data files to the target data storage space, thereby achieving the purpose of preferentially replicating the data files with smaller capacity in the source data storage space to the target data storage space, and on the premise that the replication speed of the data files is not changed, reducing the times of capturing data from the source data storage space as much as possible, and further reducing the burden of the data storage system.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 shows a schematic block diagram of an OSS system according to an embodiment of the present disclosure;

FIG. 2 shows a schematic block diagram of a data storage system according to an embodiment of the present disclosure;

FIG. 3 illustrates a flow diagram of a data replication method according to an embodiment of the present disclosure;

FIG. 4 shows a schematic block diagram of an OSS system according to an embodiment of the present disclosure;

FIG. 5 illustrates a flow chart of a data storage system control method according to an embodiment of the present disclosure;

FIG. 6 shows a block diagram of a data replication device according to an embodiment of the present disclosure;

FIG. 7 is a block diagram of a data storage system control apparatus according to an embodiment of the present disclosure;

FIG. 8 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 9 is a block diagram of a computer system suitable for use in implementing a method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.

In the present disclosure, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of labels, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to preclude the possibility that one or more other labels, numbers, steps, actions, components, parts, or combinations thereof are present or added.

It should be further noted that the embodiments and labels in the embodiments of the present disclosure may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

When the data storage system comprises a plurality of data storage spaces, in order to ensure that a user can read data which needs to be read from a target storage space. The present inventors considered the following.

In an embodiment of the present disclosure, when the target storage space does not store data requested to be read by the user, the data storage system may fetch corresponding data from the source storage space according to the configured source storage space address, and when the source storage space stores the data, the data storage system stores the data in the target storage space, so that the user can read the data from the target storage space.

Illustratively, taking an Object Storage Service (OSS) system as an example of a data Storage system, fig. 1 shows a schematic block diagram of an OSS system according to an embodiment of the present disclosure, and as shown in fig. 1, an OSS system 100 includes a overseas container (Bucket)101 at an overseas data center and a domestic container 102 at a domestic data center, where the container may be understood as a Storage space and the container is used for storing an Object (Object), where the Object is a basic unit of data stored by the OSS and the Object may be understood as data stored by the container.

The user terminal 110 may include a mobile device, a tablet computer, a laptop computer, or a built-in device in a motor vehicle, or the like, or any combination thereof. In some implementations, the smart mobile device can include a smartphone, a Personal Digital Assistant (PDA), a gaming device, a navigation device, or a point of sale (POS) device, or the like, or any combination thereof.

A user located overseas may request to read corresponding objects, i.e., data, from the OSS system 100 through the user terminal 110, and in order to reduce the time delay for the user terminal 110 to read the user-requested data, the user terminal 110 may read the corresponding data from the overseas container 101. If the overseas container 101 does not store the data requested by the user, the OSS system 100 determines a source container corresponding to the overseas container 101 according to the configured source return address, and if the determined source station is the domestic container 102, the OSS system 100 may capture the data requested by the user from the domestic container 102 by means of a source return method, such as mirroring the source return method, and if the domestic container 102 stores the data requested by the user, the OSS system may respond to the capture of the OSS system, store the data requested by the user into the overseas container 101, and return the data requested by the user to the user terminal 110 by the overseas container 101.

The disadvantages of this solution: in recent years, as the number of users using the data storage system increases, the capacity of data stored in the data storage system also increases sharply, and at the same time, users using the data storage system also spread all over the world, and in some scenarios, when a user is located overseas, the user can read corresponding data from a storage space carried by a data center located overseas in general in order to reduce the time delay for the user to read the data, but due to the high communication cost between the data center located overseas and the data center located domestically, when the storage space carried by the data center located overseas does not store data requested by the user, and when the source storage space configured is a storage space carried by the data center located domestically, the data storage system will fetch and store the data from the storage space carried by the data center located domestically into the storage space carried by the data center located overseas, considering that communication resources between a domestic data center and a overseas data center are scarce, if data with large capacity is captured in a short time, the data may occupy more communication resources, and the data capturing cost is high, the communication service provider may also be caused to forcibly limit the occupation of the communication resources by a user, that is, the data storage system is limited to capture data from a storage space borne by the domestic data center and store the data to a bandwidth of a storage space borne by the overseas data center, so that the time delay for the user to read the data is sharply increased; in addition, when the data processing resources of the data storage system are scarce, the data are frequently fetched from the source storage space and stored in the target storage space, which may increase the data processing burden of the data storage system, and may cause the data storage system to forcibly limit the number of data fetches per unit time, thereby impairing the user experience.

In view of the disadvantages of the above solutions, the inventors of the present disclosure propose a new solution: the scheme is applied to a data storage system, data in a source data storage space is copied to a target data storage space by acquiring a data capture bandwidth and a data access frequency and responding to the situation that the data capture bandwidth is larger than or equal to the target bandwidth and/or the data access frequency is larger than or equal to the target access frequency, wherein the data capture bandwidth is a bandwidth occupied when data is captured from the source data storage space to the target data storage space, the data access frequency is the number of times of capturing the data from the source data storage space in unit time, so that the data capture bandwidth and the data access frequency can reflect the number of communication resources occupied when the data is captured from the source data storage space to the target data storage space, and when the data capture bandwidth is larger than or equal to the target bandwidth, if the communication resources occupied by the data storage system are continuously increased, this may result in higher communication costs or may also result in the communication service provider forcing a limitation of the communication resources that can be used by the data storage system; when the data access frequency is greater than or equal to the target access frequency, it may be considered that if the data access frequency continues to increase, the data processing resources of the data storage system are occupied too much, which may result in an excessive data processing burden on the data storage system, and the data storage system may limit the number of data capturing times in a unit time. In order to avoid the situation that the communication cost is too high, a communication service provider forcibly limits communication resources which can be used by the data storage system and the data capturing frequency of the data storage system in unit time, the data in the source data storage space is copied to the target data storage space, so that a user can directly read the data originally stored in the source data storage space from the target data storage space without capturing the data from the source data storage space to the target data storage space, the communication cost is saved, the data processing burden of the data storage system is reduced, meanwhile, the time delay when the user reads the data is reduced, and the user experience is improved.

In order to solve the above problems, the present disclosure provides a data replication method, a data storage system control method, an apparatus, a device, and a medium. FIG. 2 shows a schematic block diagram of a data storage system according to an embodiment of the present disclosure. As shown in fig. 2, the network storage system 200 may include a target data storage space 211 and a source data storage space 212, wherein the target data storage space 211 may be carried by a first data center and the source data storage space 212 may be carried by a second data center. A data center is understood to be an entity that performs centralized processing, storage, transmission, exchange, and management of information in a physical space, and may include computer devices, server devices, network devices, storage devices, and the like.

Fig. 3 shows a flowchart of a data replication method according to an embodiment of the present disclosure, the method is applied to a data storage system, and as shown in fig. 3, the data replication method includes steps S101 and S102.

In step S101, a data capture bandwidth and a data access frequency are acquired.

The data capturing bandwidth is the bandwidth occupied when the data are captured from the source data storage space to the target data storage space, and the data access frequency is the number of times of capturing the data from the source data storage space in unit time.

In step S102, in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the target access frequency, data in the source data storage space is copied to the target data storage space.

In one embodiment of the present disclosure, the source data storage space and the target data storage space may be understood as being configured in advance, that is, one data storage space may be configured in advance as a source data storage space of a plurality of data storage spaces other than the source data storage space, and the corresponding plurality of data storage spaces other than the source data storage space is configured as a target data storage space of the source data storage space. The data stored in the target data storage space may be understood as a copy of the data stored in the source data storage space at a time.

In an embodiment of the present disclosure, the data capture bandwidth may be understood as a bandwidth occupied by the data center where the source data storage space is located to transmit the data captured from the source data storage space to the data center where the target data storage space is located. The data capture bandwidth can be detected by a data center where the source storage space is located, or detected by a data center where the target data storage space is located, or detected by other devices or systems.

In one embodiment of the present disclosure, the data access frequency may be obtained according to a data read-write log of the data storage system, where the data read-write log may include data read-write indication information, and the data read-write indication information may be used to indicate the number of times that data in a source data storage space is requested to be read in a unit time and the data is transmitted to a target data storage space, where reading a file may be regarded as being read once.

In one embodiment of the present disclosure, the target bandwidth and the target access frequency may be obtained in advance, or may be obtained from a device or other places.

In one embodiment of the present disclosure, copying data in a source data storage space to a target data storage space may be understood as copying data in the source data storage space and operations such as creation, update, and deletion of the data from the source storage space to the target storage space in an asynchronous manner.

Illustratively, taking an Object Storage Service (OSS) system as an example for explanation, fig. 4 shows a schematic block diagram of an OSS system according to an embodiment of the present disclosure, and as shown in fig. 1, an OSS system 210 includes a first data center 201 located overseas and a second data center 202 located domestically, where the first data center 201 carries an overseas container 221 and the second data center 202 carries a domestic container 222. A user terminal 203 located overseas may send a read user request data request 213 to the OSS system 210 requesting to read user request data. In order to reduce the time delay for the user side 203 to read the user request data, the user side 203 may be used to read the corresponding data from the overseas container 221. When the user request data requested by the user terminal 203 is not stored in the overseas container 221, a source container (i.e., a source storage space) corresponding to the overseas container 221 as a target container (i.e., a target storage space) may be acquired, and when it is determined that the source container corresponding to the overseas container 221 as the target container is the domestic container 222, user request data capture 232 may be performed to the domestic container 222 to capture the user request data. When the domestic container 222 includes the user requested data, the user requested data in the overseas container 221 may be stored to the overseas container 221 by mirroring back to the source 242. When the bandwidth occupied when the user request data is captured from the domestic container 222 to the overseas container 221, that is, the bandwidth occupied by mirroring back to the source is greater than or equal to the target bandwidth, and/or the number of times of capturing the user request data from the domestic container 222 within a unit time is greater than or equal to the target access frequency, the OSS system may automatically start cross-region copy 252 between the domestic container 222 and the overseas container 221, and copy the data in the domestic container 222 to the overseas container 221. After the cross-region copying 252 is completed, the data in the domestic container 222 is stored in the overseas container 221, so that when a user reads 223 the data from the overseas container 221, the data originally stored in the domestic container 222 can be directly read from the overseas container 221, and the OSS system is not required to store the user request data in the domestic container 222 to the overseas container 221 in a mirror-back source mode, so that the time delay of the user for reading the data is reduced, the communication cost of communication between the overseas container 221 and the domestic container 222 when the user reads the data is saved, and the user experience is improved.

In one embodiment of the present disclosure, in step S102, before copying the data in the source data storage space to the target data storage space in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the target access frequency, the method further includes the following steps:

In an embodiment of the present disclosure, the obtaining of the data copy configuration information may be reading the data copy configuration information stored in advance, or receiving the data copy configuration information sent by the data storage system control end or other systems or devices.

Where a data storage system control may be understood to be operated by an operator of a data storage service operator or a data storage service consumer, in some embodiments, the data storage system control may include a mobile device, a tablet computer, a laptop computer, etc., or any combination thereof. In some embodiments, the mobile device may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home devices may include smart lighting devices, control devices of smart electrical devices, smart televisions, smart cameras, or walkie talkies, or the like, or any combination thereof. In some embodiments, the wearable device may include a smart bracelet, a smart shoelace, a smart glass, a smart helmet, a smart watch, a smart garment, a smart backpack, a smart accessory, and the like, or any combination thereof. In some implementations, the smart mobile device can include a smartphone, a Personal Digital Assistant (PDA), a gaming device, a navigation device, or a point of sale (POS) device, or the like, or any combination thereof. In some embodiments, the virtual reality device and/or the augmented reality device may include a virtual reality helmet, virtual reality glass, a virtual reality patch, an augmented reality helmet, augmented reality glass, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device and/or augmented reality device may include various virtual reality products and the like.

In one embodiment of the present disclosure, the first data storage space indication and the second data storage space indication may be understood as indicating respective corresponding data storage spaces. Further, the first data storage space indication and the second data storage space indication may also be used to indicate the region to which the respective corresponding data storage space belongs.

In an embodiment of the present disclosure, before copying the data in the source data storage space to the target data storage space in step S102, the method further includes the following steps:

In one embodiment of the present disclosure, capturing a data capture history of data from the source data storage space may be understood as a data read log recording at least one of a data read time, a data read frequency, and a data read source of data read from the source data storage space, and it may be determined whether corresponding data in the source data storage space has been captured according to the data capture history.

In one embodiment of the present disclosure, the data crawling history includes an accessed time of a data directory in which the data is located in the source data storage space;

determining uncaptured target data in a source data storage space according to the data capture history, which can be realized by the following steps:

In one embodiment of the present disclosure, the target access time may be understood as a time at which to begin fetching data from the source data storage space to the target data storage space, or may also be understood as a time before beginning to fetch data from the source data storage space to the target data storage space. When the latest access time of a certain data directory is earlier than the target access time, it can be considered that the data directory is not accessed after the data directory starts to be fetched from the source data storage space to the target data storage space, and therefore the data in the data directory is not fetched to the target data storage space.

In one embodiment of the present disclosure, the target access time may be obtained in response to a data fetch from the source data storage space to the target data storage space, or may be obtained from another device or system.

In an embodiment of the present disclosure, the obtaining of the capacity of the data file in the source data storage space may be reading previously obtained data file attribute information, and obtaining the capacity of the data file in the source data storage space according to the data file attribute information, where the data file attribute information is data file attribute information indicating respective capacities of the data files in the source data storage space; the method can also be used for receiving the data file attribute information sent by a data center or other devices or systems bearing the source data storage space, and acquiring the capacity of the data file in the source data storage space according to the data file attribute information.

Fig. 5 is a flowchart illustrating a data storage system control method according to an embodiment of the disclosure, which is applied to a data storage system control side.

Where a data storage system control may be understood to be operated by an operator of a data storage service operator or a data storage service consumer, in some embodiments, the data storage system control may include a mobile device, a tablet computer, a laptop computer, etc., or any combination thereof. In some embodiments, the mobile device may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home devices may include smart lighting devices, control devices of smart electrical devices, smart televisions, smart cameras, or walkie talkies, or the like, or any combination thereof. In some embodiments, the wearable device may include a smart bracelet, a smart lace, smart glass, a smart helmet, a smart watch, a smart garment, a smart backpack, a smart accessory, and the like, or any combination thereof. In some implementations, the smart mobile device can include a smartphone, a Personal Digital Assistant (PDA), a gaming device, a navigation device, or a point of sale (POS) device, or the like, or any combination thereof. In some embodiments, the virtual reality device and/or the augmented reality device may include a virtual reality helmet, virtual reality glass, a virtual reality patch, an augmented reality helmet, augmented reality glass, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device and/or augmented reality device may include various virtual reality products and the like.

As shown in fig. 5, the data storage system control method includes steps S201 and S202.

In step S201, data copy configuration information is acquired.

The data replication configuration information comprises a first data storage space mark, a second data storage space mark and threshold configuration information, and the threshold configuration information comprises at least one of configuration bandwidth and configuration access frequency.

In step S202, data copy configuration information is transmitted.

In an embodiment of the present disclosure, the data replication configuration information may be stored in advance, or may be input through a human-computer interaction device at a control end of the data storage system. The data copying configuration information is input through a human-computer interaction device at a control end of the data storage system, and the data copying configuration information input by a user can be acquired through at least one of a keyboard, a touch screen and a touch pad at the control end of the data storage system; or acquiring voice information input by a user through a microphone at a control end of the data storage system, and performing voice recognition on the voice information to acquire data copy configuration information.

In one embodiment of the present disclosure, the sending of the data replication configuration information may be sending the data replication configuration information to the data storage system through a wired network or a wireless network.

In one embodiment of the present disclosure, the data replication configuration information further includes at least one of first data replication indication information and second data replication indication information;

A data replication device according to an embodiment of the present disclosure is described below with reference to fig. 6, and fig. 6 illustrates a block diagram of the structure of the data replication device according to an embodiment of the present disclosure.

As shown in fig. 6, the data replication device 300 includes:

a bandwidth frequency obtaining module 301, configured to obtain a data capture bandwidth and a data access frequency, where the data capture bandwidth is a bandwidth occupied by a target data storage space when capturing data from a source data storage space, and the data access frequency is a number of times of capturing data from the source data storage space in a unit time;

a data replication module 302 configured to replicate data in the source data storage space to the target data storage space in response to the data capture bandwidth being greater than or equal to the target bandwidth and/or the data access frequency being greater than or equal to the target access frequency.

According to the technical scheme provided by the embodiment of the disclosure, data in the source data storage space is copied to the target data storage space by acquiring the data capture bandwidth and the data access frequency and responding to the situation that the data capture bandwidth is greater than or equal to the target bandwidth and/or the data access frequency is greater than or equal to the target access frequency, wherein the data capture bandwidth is a bandwidth occupied when capturing data from the source data storage space to the target data storage space, and the data access frequency is the number of times of capturing data from the source data storage space in a unit time, so that the data capture bandwidth and the data access frequency can reflect the amount of communication resources occupied when capturing data from the source data storage space to the target data storage space, and when the data capture bandwidth is greater than or equal to the target bandwidth, it can be considered that if the communication resources occupied by the data storage system are continuously increased, this may result in higher communication costs or may also result in the communication service provider forcing a limitation of the communication resources that can be used by the data storage system; when the data access frequency is greater than or equal to the target access frequency, it may be considered that if the data access frequency continues to increase, the data processing resources of the data storage system are occupied too much, which may result in an excessive data processing burden on the data storage system, and the data storage system may limit the number of data capturing times in a unit time. In order to avoid the situation that the communication cost is too high, a communication service provider forcibly limits communication resources which can be used by the data storage system and the data capturing frequency of the data storage system in unit time, the data in the source data storage space is copied to the target data storage space, so that a user can directly read the data originally stored in the source data storage space from the target data storage space without capturing the data from the source data storage space to the target data storage space, the communication cost is saved, the data processing burden of the data storage system is reduced, meanwhile, the time delay when the user reads the data is reduced, and the user experience is improved.

It will be appreciated by a person skilled in the art that the solution described with reference to fig. 6 can be combined with any of the embodiments described above to achieve the technical effects achieved by any of the embodiments described above. For details, reference may be made to the description of the foregoing embodiments, and details thereof are not described herein again.

A data storage system control device according to an embodiment of the present disclosure is described below with reference to fig. 7, and fig. 7 shows a block diagram of the data storage system control device according to an embodiment of the present disclosure.

As shown in fig. 7, the data storage system control device 400 includes:

a configuration information obtaining module 401 configured to obtain data replication configuration information, where the data replication configuration information includes a first data storage space indicator, a second data storage space indicator, and threshold configuration information, and the threshold configuration information includes at least one of a configuration bandwidth and a configuration access frequency;

a configuration information sending module 402 configured to send data replication configuration information.

It will be appreciated by a person skilled in the art that the solution described with reference to fig. 7 may be combined with any of the embodiments described above to achieve the technical effects achieved by any of the embodiments described above. For details, reference may be made to the description of the foregoing embodiments, and details thereof are not described herein again.

Fig. 8 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

The disclosed embodiment also provides an electronic device, as shown in fig. 8, the electronic device 500 includes at least one processor 501; and a memory 502 communicatively coupled to the at least one processor 501; wherein the memory 502 stores instructions executable by the at least one processor 501, the instructions being executable by the at least one processor 501 to implement the steps of:

and sending the data replication configuration information.

the first data copying indication information is used for indicating to acquire data capture history of data captured from the source data storage space and determining target data which are not captured in the source data storage space according to the data capture history;

As shown in fig. 9, the computer system 600 includes a processing unit 601 which can execute various processes in the embodiments shown in the above-described drawings in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary. The processing unit 601 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.

In particular, according to embodiments of the present disclosure, the methods described above with reference to the figures may be implemented as computer software programs. Illustratively, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the method of the figures. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609 and/or installed from the removable medium 611. Illustratively, embodiments of the present disclosure include a readable storage medium having stored thereon computer instructions which, when executed by a processor, implement program code for performing the methods of the figures.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. Illustratively, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.

As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the node in the above embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept. Exemplary above features are disclosed in this disclosure (but not limited to) as technical features with similar functions are replaced by each other.

Claims

1. A data replication method, wherein the method is applied to a data storage system, the method comprising:

in response to the data capture bandwidth being greater than or equal to a target bandwidth and/or the data access frequency being greater than or equal to a target access frequency, copying data in the source data storage space to the target data storage space.

2. The data replication method of claim 1, wherein prior to replicating data in the source data storage space to the target data storage space in response to the data capture bandwidth being greater than or equal to a target bandwidth and/or the data access frequency being greater than or equal to an access frequency threshold, the method further comprises:

determining the configured bandwidth as the target bandwidth in response to the first data storage space indicating that the indicated data storage space is the source data storage space and the second data storage space indicating that the indicated data storage space is the target data storage space;

3. The data replication method of claim 1 or 2, wherein prior to said replicating data in said source data storage space to said target data storage space, said method further comprises:

acquiring data capture history of data captured from the source data storage space;

determining uncaptured target data in the source data storage space according to the data capture history;

the copying the data in the source data storage space to the target data storage space comprises:

copying the target data from the source data storage space to the target data storage space.

4. The data replication method of claim 3, wherein the data crawling history comprises an accessed time of a data directory in which data in the source data storage space is located;

the determining the target data which is not grabbed in the source data storage space according to the data grabbing history comprises:

and determining a target data directory of which the last access time of the source data storage space is earlier than the target access time, and determining data in the target data directory as the target data.

5. The data replication method of claim 1 or 2, wherein prior to said replicating data in said source data storage space to said target data storage space, said method further comprises:

acquiring the capacity of the data files in the source data storage space, and sequencing the data files in the source data storage space according to the capacity in a sequence from small to large;

and copying the sequenced data files to the target data storage space in sequence.

6. A data storage system control method is applied to a data storage system control terminal, and comprises the following steps:

and sending the data replication configuration information.

7. The data storage system control method of claim 6, wherein the data replication configuration information further comprises at least one of first data replication indication information, second data replication indication information;

the first data copying indication information is used for indicating to acquire data capturing history of data captured from a source data storage space, and determining target data which are not captured in the source data storage space according to the data capturing history;

the second data replication indication information is used for indicating to acquire the capacity of the data files in the source data storage space, and sorting the data files in the source data storage space according to the capacity in a sequence from small to large.

8. A data copying apparatus, comprising:

a data replication module configured to replicate data in the source data storage space to a target data storage space in response to the data capture bandwidth being greater than or equal to a target bandwidth and/or the data access frequency being greater than or equal to a target access frequency.

9. A data storage system control apparatus, wherein the apparatus comprises:

a configuration information obtaining module configured to obtain data replication configuration information, where the data replication configuration information includes a first data storage space indicator, a second data storage space indicator, and threshold configuration information, and the threshold configuration information includes at least one of a configuration bandwidth and a configuration access frequency;

a configuration information sending module configured to send the data replication configuration information.

10. An electronic device comprising a memory and at least one processor; wherein the memory is to store one or more computer instructions, wherein the one or more computer instructions are to be executed by the at least one processor to implement the method steps of any one of claims 1-7.

11. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the method steps of any of claims 1-7.