CN111897494A - Target file processing method and device - Google Patents
Target file processing method and device Download PDFInfo
- Publication number
- CN111897494A CN111897494A CN202010734088.5A CN202010734088A CN111897494A CN 111897494 A CN111897494 A CN 111897494A CN 202010734088 A CN202010734088 A CN 202010734088A CN 111897494 A CN111897494 A CN 111897494A
- Authority
- CN
- China
- Prior art keywords
- node
- processing
- target file
- determining
- processing range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 6
- 238000012545 processing Methods 0.000 claims abstract description 138
- 238000011084 recovery Methods 0.000 claims abstract description 70
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000005012 migration Effects 0.000 claims abstract description 31
- 238000013508 migration Methods 0.000 claims abstract description 31
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 14
- 230000000694 effects Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for processing a target file. Wherein, the method comprises the following steps: acquiring a processing range associated with each node in the disaster recovery cluster; determining a preset value of a target file; determining nodes related to the processing range in which the preset value is positioned; and migrating the target file according to the node. The invention solves the technical problems that in the related technology, only single-point resources can be used for data migration, the migration is finished after the node is down, and the migration speed is slow.
Description
Technical Field
The invention relates to the field of memory data processing, in particular to a method and a device for processing a target file.
Background
With the popularity of distributed storage and software defined storage, data centers, more and more users need to store historical data from a traditional single point: local storage, IPSAN, NAS, etc. migrate to the new distributed infrastructure.
Helping users to efficiently and completely migrate business data from existing storage to other storage: object or NAS, migration tool also grows as needed.
However, the existing file or data migration method adopts data migration of a single point of resource, and the migration method has the following disadvantages:
1) the single point of failure, after the node running the migration is down, the migration is finished;
2) only single point resources can be used, and data relocation is slow due to CPU/memory limitations.
Aiming at the problems that in the related art, data migration can only be performed by using a single-point resource, migration is finished after a node is down, and the migration speed is slow, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a method and a device for processing a target file, which are used for at least solving the technical problems that data migration can be performed only by using a single-point resource, the migration is finished after a node is down and the migration speed is slow in the related art.
According to an aspect of the embodiments of the present invention, there is provided a method for processing a target file, including: acquiring a processing range associated with each node in the disaster recovery cluster; determining a preset value of a target file; determining nodes associated with the processing range in which the preset value is positioned; and carrying out migration processing on the target file according to the node.
Optionally, before obtaining the processing range associated with each node in the disaster recovery cluster, the method further includes: constructing the disaster recovery cluster, wherein the disaster recovery cluster comprises a plurality of nodes, each node is associated with a different processing range, and the processing range comprises at least one of the following: hash range, MD5 range.
Optionally, constructing the disaster recovery cluster includes: acquiring the node number and/or the node weight of each node; determining a processing range corresponding to the node number and/or the node weight; and associating the node number and/or the node weight with a processing range corresponding to the node number and/or the node weight to generate the disaster recovery cluster.
Optionally, determining the predetermined value of the target file comprises: acquiring a file name and/or a relative path of the target file; generating a predetermined value of the target file according to the file name and/or the relative path, wherein the predetermined value comprises at least one of the following: hash value, MD5 value.
Optionally, determining the node associated with the processing range in which the predetermined value is located includes: judging the processing range of the preset value to obtain a judgment result; and determining the nodes associated with the processing range according to the judgment result.
Optionally, migrating the target file according to the node includes: and writing the target file into a specified storage position.
Optionally, the method further comprises: and under the condition that any node in the disaster recovery cluster is offline, dynamically adjusting the processing range associated with other nodes.
According to another aspect of the embodiments of the present invention, there is also provided a device for processing a target file, including: the acquisition module is used for acquiring a processing range associated with each node in the disaster recovery cluster; the first determining module is used for determining a preset value of the target file; the second determining module is used for determining the nodes related to the processing range where the preset value is located; and the processing module is used for carrying out migration processing on the target file according to the node.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and when the program runs, the apparatus where the computer-readable storage medium is located is controlled to execute the method for processing the target file according to any one of the above.
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, where the program executes a processing method of an object file according to any one of the foregoing.
In the embodiment of the invention, the processing range associated with each node in the disaster recovery cluster is obtained; determining a preset value of a target file; determining nodes associated with the processing range in which the preset value is positioned; according to the mode of migrating the target file by the nodes, the target file is migrated by the nodes in the disaster recovery cluster, so that the purpose that one disaster recovery cluster is maintained by multiple nodes and one task is processed by the multiple nodes together is achieved, the technical effects of avoiding single-point faults and improving the processing speed of the disaster recovery task are achieved, and the technical problems that in the related technology, data migration can be performed only by using single-point resources, the migration is finished after the nodes are crashed and the migration speed is slow are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow chart of a method of processing a target file according to an embodiment of the present invention;
FIG. 2 is a flow diagram of processing logic for each node in a disaster recovery cluster according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a target file processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided an embodiment of a method for processing an object file, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than that herein.
Fig. 1 is a flowchart of a method for processing an object file according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, acquiring a processing range associated with each node in the disaster recovery cluster;
the disaster recovery cluster includes a plurality of nodes, wherein each node is associated with a different processing scope. Alternatively, the treatment scope includes, but is not limited to: hash range, MD5 range.
Step S104, determining a preset value of the target file;
the target file is a file under a data directory to be migrated; the predetermined values of the target file include, but are not limited to: hash value, MD5 value. It should be noted that the predetermined value of the target file may be one or more, and is not limited herein.
In a specific implementation process, a certain corresponding relationship exists between a preset value of a target file and different processing range disaster recovery clusters associated with each node. For example, if each node in the disaster recovery cluster is associated with a hash range, the predetermined value of the corresponding target file is a hash value; and each node in the disaster recovery cluster is associated with the MD5 range, and the corresponding preset value of the target file is the MD5 value. Of course, each node in the disaster recovery cluster may be associated with both the hash range and the MD5 range, and the predetermined value of the target file may be flexibly adjusted according to the actual application.
Step S106, determining nodes related to the processing range where the preset value is located;
as an optional embodiment, for example, the disaster recovery cluster at least includes a first node, a second node, a third node, and a fourth node, where the first node is associated with a first processing range, the second node is associated with a second processing range, the third node is associated with a third processing range, and the fourth node is associated with a fourth processing range. Further, if the predetermined value of the target file is within the first processing range, the associated node is the first node, and if the predetermined value of the target file is within the second processing range, the associated node is the second node, and the third node and the fourth node may also be obtained in the same manner.
And step S108, migrating the target file according to the nodes.
In a specific implementation process, if the target file has a plurality of predetermined values and the predetermined values are in different processing ranges, different nodes associated with the processing ranges where the different predetermined values are located can be determined, and then the target file is migrated through the different nodes.
Through the steps, the target file can be migrated through the multiple nodes in the disaster recovery cluster, the purposes that one disaster recovery cluster is maintained by multiple nodes and one task is processed by the multiple nodes together are achieved, the technical effects of avoiding single-point faults and improving the processing speed of the disaster recovery task are achieved, and the technical problems that in the related technology, data migration can be completed only by using single-point resources, the migration is finished after the nodes are down and the migration speed is slow are solved.
Optionally, before obtaining the processing range associated with each node in the disaster recovery cluster, the method further includes: the method comprises the following steps of constructing a disaster tolerant cluster, wherein the disaster tolerant cluster comprises a plurality of nodes, each node is associated with a different processing range, and the processing range comprises at least one of the following: hash range, MD5 range.
As an optional embodiment, by constructing a disaster recovery cluster of multiple nodes, one disaster recovery cluster can be maintained by multiple nodes, and a single point of failure is effectively avoided.
It should be noted that each node in the disaster recovery cluster is associated with a different processing range, where the processing range may be a hash range, an MD5 range, or other processing ranges, and is not described herein any more.
Optionally, constructing the disaster tolerant cluster includes: acquiring the node number and/or the node weight of each node; determining a processing range corresponding to the node number and/or the node weight; and associating the node number and/or the node weight with a processing range corresponding to the node number and/or the node weight to generate the disaster recovery cluster.
As an alternative embodiment, the node number of each node may be obtained; determining a processing range corresponding to the node number; and further associating the node number with the processing range corresponding to the node number to generate the disaster recovery cluster. The node weight of each node can be obtained; determining a processing range corresponding to the node weight; and further associating the node weight and the processing range corresponding to the node weight to generate the disaster recovery cluster. In addition, the above two embodiments can be combined, and are not described herein again.
It should be noted that the node number includes an ID of the node; the node weights may be configured according to hardware resources. For example, the more hardware resources corresponding to a node, the greater the node weight; the less hardware resources corresponding to the node, the smaller the node weight.
By the implementation mode, the mode for generating the disaster recovery cluster can be optimized, the mode is more diversified, and the disaster recovery cluster meeting the application requirements can be generated.
Optionally, determining the predetermined value of the target file comprises: acquiring a file name and/or a relative path of a target file; and generating a preset value of the target file according to the file name and/or the relative path, wherein the preset value comprises at least one of the following values: hash value, MD5 value.
As an alternative example, the file name of the target file may be obtained, and then the predetermined value of the target file is generated according to the file name, or the relative path of the target file may be obtained, and then the predetermined value of the target file is generated according to the relative path, or the two above-mentioned embodiments may be combined together. Further, in any of the above-described manners, not only the hash value of the target file but also the MD5 value of the target file may be generated. In the specific implementation process, the preset value for generating the target file can be flexibly selected.
Optionally, the determining a node associated with the processing range in which the predetermined value is located includes: judging the processing range of the preset value to obtain a judgment result; and determining the nodes associated with the processing range according to the judgment result.
As an alternative embodiment, the processing range may be multiple, and it may be determined in which processing range the predetermined value of the target file is located. For example, the processing range may include a first processing range, a second processing range, a third processing range, and a fourth processing range, and if the predetermined value is in the second processing range, the node associated with the second processing range is determined according to the determination result. Through the implementation mode, the nodes related to the processing range can be accurately and quickly determined.
Optionally, migrating the target file according to the node includes: and writing the target file into the specified storage position.
As an alternative embodiment, during the migration process of the target file according to different nodes, the target file may be written into the designated storage location by different nodes. The storage location includes, but is not limited to, the memory of the target.
Optionally, the method further includes: and under the condition that any node in the disaster recovery cluster is offline, dynamically adjusting the processing range associated with other nodes.
As an optional embodiment, in the process of migrating the target file, in order to more effectively avoid the adverse effect of a single point of failure, the processing range associated with other nodes may be dynamically adjusted when any node in the disaster recovery cluster is offline. Specifically, dynamically adjusting the processing range associated with the other nodes includes: determining an online node in the disaster recovery cluster; and reallocating the processing scope associated with the online node. By the mode, the processing range of the related node can be adjusted in time, and single-point faults are effectively avoided.
An alternative embodiment of the invention is described below.
As an optional embodiment, a disaster recovery cluster may be constructed, taking a disaster recovery cluster with 4 nodes as an example. It should be noted that the number of other nodes (for example, 6 nodes, 8 nodes, etc.) is also similar logic, and is not described in detail.
The hash range 0-0 xffffffffff is distributed to a plurality of nodes, and the weight can be set according to hardware resource configuration. In this embodiment, the weight of each node is the same:
the node A is responsible for files with hash ranges of 0-0x 3fffff9 a;
the node B is responsible for files with hash ranges from 0x3fffff9B to 0x7fffff 36;
the C node is responsible for files with hash ranges of 0x7fffff 37-0 xbfffded 2;
the node D is responsible for files with hash ranges between 0xbfffed 3-0 xffffffff.
Fig. 2 is a flowchart of processing logic of each node in the disaster recovery cluster according to an embodiment of the present invention, and as shown in fig. 2, each node in the disaster recovery cluster may perform file migration according to such processing logic.
In the above embodiment, a plurality of nodes maintain a disaster recovery cluster, and collectively process a disaster recovery task, thereby avoiding a single point of failure. When a node is offline, the hash responsible for the node can be dynamically adjusted, that is, the hash range is redistributed to other online nodes.
In addition, the multiple nodes cooperatively process one disaster recovery task, and each node is responsible for a section of range, so that all files in the disaster recovery task are completely processed, and the processed files between the two nodes are not conflicted. The multiple nodes process one task together, so that hardware resources of the multiple nodes can be shared, and the processing speed of the disaster recovery task is increased.
In a specific implementation process, the complete hash range may be other ranges, and may be adjusted as needed.
As an alternative embodiment, the full hash range may be replaced by a full MD5 range, where the full MD5 range includes: 00-ff, each node obtains a section of range according to the weight, then the MD5 value of the file name is calculated, and if two characters at the end of the MD5 value are in the processing range of the node, the file is processed by the node.
In the above embodiment, the hash value or the MD5 value may be calculated as the file name or the relative path of the file.
Example 2
According to another aspect of the embodiment of the present invention, there is also provided an apparatus for processing an object file, and fig. 3 is a schematic diagram of an apparatus for processing an object file according to an embodiment of the present invention, as shown in fig. 3, the apparatus for processing an object file includes: an acquisition module 32, a first determination module 34, a second determination module 36, and a processing module 38. The processing means for the object file will be described in detail below.
An obtaining module 32, configured to obtain a processing range associated with each node in the disaster recovery cluster; a first determining module 34, connected to the acquiring module 32, for determining a predetermined value of the target file; a second determining module 36, connected to the first determining module 34, for determining a node associated with a processing range in which a predetermined value is located; and the processing module 38 is connected to the second determining module 36, and is configured to perform migration processing on the target file according to the node.
The disaster recovery cluster includes a plurality of nodes, wherein each node is associated with a different processing scope. Alternatively, the treatment scope includes, but is not limited to: hash range, MD5 range.
The target file is a file under a data directory to be migrated; the predetermined values of the target file include, but are not limited to: hash value, MD5 value. It should be noted that the predetermined value of the target file may be one or more, and is not limited herein.
In a specific implementation process, a certain corresponding relationship exists between a preset value of a target file and different processing range disaster recovery clusters associated with each node. For example, if each node in the disaster recovery cluster is associated with a hash range, the predetermined value of the corresponding target file is a hash value; and each node in the disaster recovery cluster is associated with the MD5 range, and the corresponding preset value of the target file is the MD5 value. Of course, each node in the disaster recovery cluster may be associated with both the hash range and the MD5 range, and the predetermined value of the target file may be flexibly adjusted according to the actual application.
As an optional embodiment, for example, the disaster recovery cluster at least includes a first node, a second node, a third node, and a fourth node, where the first node is associated with a first processing range, the second node is associated with a second processing range, the third node is associated with a third processing range, and the fourth node is associated with a fourth processing range. Further, if the predetermined value of the target file is within the first processing range, the associated node is the first node, and if the predetermined value of the target file is within the second processing range, the associated node is the second node, and the third node and the fourth node may also be obtained in the same manner.
In a specific implementation process, if the target file has a plurality of predetermined values and the predetermined values are in different processing ranges, different nodes associated with the processing ranges where the different predetermined values are located can be determined, and then the target file is migrated through the different nodes.
It should be noted that the above modules may be implemented by software or hardware, for example, for the latter, the following may be implemented: the modules can be located in the same processor; alternatively, the modules may be located in different processors in any combination.
It should be noted here that the above-mentioned obtaining module 32, the first determining module 34, the second determining module 36 and the processing module 38 correspond to steps S102 to S108 in embodiment 1, and the above-mentioned modules are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to what is disclosed in embodiment 1 above. It should be noted that the modules described above as part of an apparatus may be implemented in a computer system such as a set of computer-executable instructions.
As can be seen from the above, in the above embodiments of the present application, the target file is migrated through the multiple nodes in the disaster recovery cluster, so that the multiple nodes maintain one disaster recovery cluster and collectively process one task, thereby achieving the technical effects of avoiding a single point fault and increasing the processing speed of the disaster recovery task, and further solving the technical problems that in the related art, data migration can be performed only by using a single point resource, which causes a downtime of a node, migration is completed, and the migration speed is slow.
Optionally, before obtaining the processing range associated with each node in the disaster recovery cluster, the apparatus further includes: the system comprises a building module and a processing module, wherein the building module is used for building a disaster recovery cluster, the disaster recovery cluster comprises a plurality of nodes, each node is associated with a different processing range, and the processing range comprises at least one of the following: hash range, MD5 range.
Optionally, the building module includes: a first obtaining unit, configured to obtain a node number and/or a node weight of each node; the first determining unit is used for determining a processing range corresponding to the node number and/or the node weight; and the first generating unit is used for associating the node number and/or the node weight with the processing range corresponding to the node number and/or the node weight to generate the disaster recovery cluster.
Optionally, the first determining module includes: the second acquisition unit is used for acquiring the file name and/or the relative path of the target file; the second generating unit is used for generating a preset value of the target file according to the file name and/or the relative path, wherein the preset value comprises at least one of the following values: hash value, MD5 value.
Optionally, the second determining module includes: the judging unit is used for judging the processing range of the preset value to obtain a judging result; and the second determining unit is used for determining the nodes related to the processing range according to the judgment result.
Optionally, the processing module includes: and the writing unit is used for writing the target file into the specified storage position.
Optionally, the apparatus further comprises: and the adjusting module is used for dynamically adjusting the processing range associated with other nodes under the condition that any node in the disaster recovery cluster is offline.
Example 3
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and when the program runs, the apparatus where the computer-readable storage medium is located is controlled to execute the method for processing the object file in any one of the above.
Optionally, in this embodiment, the computer-readable storage medium may be located in any one of a group of computer terminals in a computer network or in any one of a group of mobile terminals, and the computer-readable storage medium includes a stored program.
Optionally, the program when executed controls an apparatus in which the computer-readable storage medium is located to perform the following functions: acquiring a processing range associated with each node in the disaster recovery cluster; determining a preset value of a target file; determining nodes related to the processing range in which the preset value is positioned; and migrating the target file according to the node.
Example 4
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes a processing method of an object file in any one of the above.
The embodiment of the application provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program to realize the following steps: acquiring a processing range associated with each node in the disaster recovery cluster; determining a preset value of a target file; determining nodes related to the processing range in which the preset value is positioned; and migrating the target file according to the node.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: acquiring a processing range associated with each node in the disaster recovery cluster; determining a preset value of a target file; determining nodes related to the processing range in which the preset value is positioned; and migrating the target file according to the node.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for processing a target file, comprising:
acquiring a processing range associated with each node in the disaster recovery cluster;
determining a preset value of a target file;
determining nodes associated with the processing range in which the preset value is positioned;
and carrying out migration processing on the target file according to the node.
2. The method of claim 1, wherein prior to obtaining the processing scope associated with each node in the disaster recovery cluster, the method further comprises:
constructing the disaster recovery cluster, wherein the disaster recovery cluster comprises a plurality of nodes, each node is associated with a different processing range, and the processing range comprises at least one of the following: hash range, MD5 range.
3. The method of claim 2, wherein constructing the disaster tolerant cluster comprises:
acquiring the node number and/or the node weight of each node;
determining a processing range corresponding to the node number and/or the node weight;
and associating the node number and/or the node weight with a processing range corresponding to the node number and/or the node weight to generate the disaster recovery cluster.
4. The method of claim 1, wherein determining the predetermined value for the target file comprises:
acquiring a file name and/or a relative path of the target file;
generating a predetermined value of the target file according to the file name and/or the relative path, wherein the predetermined value comprises at least one of the following: hash value, MD5 value.
5. The method of claim 1, wherein determining the node associated with the processing range in which the predetermined value is located comprises:
judging the processing range of the preset value to obtain a judgment result;
and determining the nodes associated with the processing range according to the judgment result.
6. The method of claim 1, wherein migrating the target file according to the node comprises:
and writing the target file into a specified storage position.
7. The method according to any one of claims 1 to 6, further comprising:
and under the condition that any node in the disaster recovery cluster is offline, dynamically adjusting the processing range associated with other nodes.
8. An apparatus for processing an object document, comprising:
the acquisition module is used for acquiring a processing range associated with each node in the disaster recovery cluster;
the first determining module is used for determining a preset value of the target file;
the second determining module is used for determining the nodes related to the processing range where the preset value is located;
and the processing module is used for carrying out migration processing on the target file according to the node.
9. A computer-readable storage medium, comprising a stored program, wherein when the program runs, the program controls a device on which the computer-readable storage medium is located to execute the processing method of the object file according to any one of claims 1 to 7.
10. A processor, configured to execute a program, wherein the program executes to perform the processing method of the object file according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010734088.5A CN111897494A (en) | 2020-07-27 | 2020-07-27 | Target file processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010734088.5A CN111897494A (en) | 2020-07-27 | 2020-07-27 | Target file processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111897494A true CN111897494A (en) | 2020-11-06 |
Family
ID=73190226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010734088.5A Pending CN111897494A (en) | 2020-07-27 | 2020-07-27 | Target file processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111897494A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033774A1 (en) * | 2003-08-05 | 2005-02-10 | James Brentano | System and method for bulk transfer of digital goods |
| CN105468473A (en) * | 2014-07-16 | 2016-04-06 | 北京奇虎科技有限公司 | Data migration method and data migration apparatus |
| CN105760240A (en) * | 2014-12-16 | 2016-07-13 | 航天信息股份有限公司 | Distributed task processing method and device |
| CN105760480A (en) * | 2016-02-15 | 2016-07-13 | 深圳市中博科创信息技术有限公司 | Distribution algorithm for cluster file system |
| CN105930545A (en) * | 2016-06-29 | 2016-09-07 | 浙江宇视科技有限公司 | Method and device for migrating files |
| CN106021462A (en) * | 2016-05-17 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method of cluster file system and cluster file system |
| CN109656911A (en) * | 2018-12-11 | 2019-04-19 | 江苏瑞中数据股份有限公司 | Distributed variable-frequencypump Database Systems and its data processing method |
-
2020
- 2020-07-27 CN CN202010734088.5A patent/CN111897494A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033774A1 (en) * | 2003-08-05 | 2005-02-10 | James Brentano | System and method for bulk transfer of digital goods |
| CN105468473A (en) * | 2014-07-16 | 2016-04-06 | 北京奇虎科技有限公司 | Data migration method and data migration apparatus |
| CN105760240A (en) * | 2014-12-16 | 2016-07-13 | 航天信息股份有限公司 | Distributed task processing method and device |
| CN105760480A (en) * | 2016-02-15 | 2016-07-13 | 深圳市中博科创信息技术有限公司 | Distribution algorithm for cluster file system |
| CN106021462A (en) * | 2016-05-17 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method of cluster file system and cluster file system |
| CN105930545A (en) * | 2016-06-29 | 2016-09-07 | 浙江宇视科技有限公司 | Method and device for migrating files |
| CN109656911A (en) * | 2018-12-11 | 2019-04-19 | 江苏瑞中数据股份有限公司 | Distributed variable-frequencypump Database Systems and its data processing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10747780B2 (en) | Blockchain-based data processing method and device | |
| CN106331098B (en) | Server cluster system | |
| US20200119988A1 (en) | Optimizing application configurations in a provider network | |
| CN107621973B (en) | Cross-cluster task scheduling method and device | |
| CN108023967B (en) | Data balancing method and device and management equipment in distributed storage system | |
| EP3786798A1 (en) | Computing connected components in large graphs | |
| EP2787446A1 (en) | Distributed storage method, device and system | |
| EP3442201B1 (en) | Cloud platform construction method and cloud platform | |
| CN111309443B (en) | Virtualization method, device, equipment and storage medium of Internet of things equipment system | |
| CN113687964B (en) | Data processing method, device, electronic equipment, storage medium and program product | |
| CN106775470B (en) | Data storage method and system | |
| CN109145053B (en) | Data processing method and device, client and server | |
| CN111666266A (en) | Data migration method and related equipment | |
| CN105978938A (en) | Service processing equipment service status determining method and scheduling equipment | |
| CN107656796B (en) | Virtual machine cold migration method, system and equipment | |
| CN108270753B (en) | Method and device for logging out user account | |
| CN111130834B (en) | Method and device for processing network elasticity strategy | |
| CN103902401A (en) | Virtual machine fault tolerance method and device based on monitoring | |
| EP3349416B1 (en) | Relationship chain processing method and system, and storage medium | |
| CN108228310B (en) | Balanced deployment method and device of virtual network function | |
| WO2018001375A1 (en) | Physical to virtual migration method, physical server, virtual server, and system | |
| CN105487946A (en) | Fault computer automatic switching method and device | |
| CN109753782B (en) | Adaptive processing method and device for cloud security resource pool | |
| CN109815047B (en) | Data processing method and related device | |
| CN111897494A (en) | Target file processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100094 101, floors 1-5, building 7, courtyard 3, fengxiu Middle Road, Haidian District, Beijing Applicant after: Beijing Xingchen Tianhe Technology Co.,Ltd. Address before: 100097 room 806-1, block B, zone 2, Jinyuan times shopping center, indigo factory, Haidian District, Beijing Applicant before: XSKY BEIJING DATA TECHNOLOGY Corp.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201106 |
|
| RJ01 | Rejection of invention patent application after publication |