CN115129739B - Data file breakpoint continuous transmission method, device, equipment and storage medium - Google Patents
Data file breakpoint continuous transmission method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115129739B CN115129739B CN202211058208.XA CN202211058208A CN115129739B CN 115129739 B CN115129739 B CN 115129739B CN 202211058208 A CN202211058208 A CN 202211058208A CN 115129739 B CN115129739 B CN 115129739B
- Authority
- CN
- China
- Prior art keywords
- file
- database
- data
- acquiring
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Abstract
The invention relates to a method, a device, equipment and a storage medium for breakpoint continuous transmission of a data file, wherein the method comprises the following steps: when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to the first database; acquiring an intermediate control file corresponding to the second database according to the target control file; when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to the first database and a second control file corresponding to the second database; acquiring a middle control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database; and acquiring a target data file set corresponding to the second database according to the intermediate control file, wherein the number of data files in the target data file set is consistent with the first data volume, so that the problem that the data files in a plurality of databases cannot be updated synchronously and the safety of the data files is influenced can be avoided.
Description
Technical Field
The invention relates to the technical field of database processing, in particular to a data file breakpoint continuous transmission method, a data file breakpoint continuous transmission device, data file breakpoint continuous transmission equipment and a storage medium.
Background
With the development of the big data era, the number of data files of the database is also larger and larger. In order to ensure the security and stability of data files, it is a necessary trend to store data files by using a plurality of databases.
However, when the number of data files in the databases increases, and when an abnormal condition occurs in updating the data files in the databases, for example, a network is interrupted, the updating of the data files in the databases cannot be synchronized, which affects the security and stability of the data files, and further affects the working efficiency.
Disclosure of Invention
Aiming at the technical problem, the invention protects a data file breakpoint continuous transmission method, which comprises the following steps:
the method comprises the steps of obtaining a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, the second data volume is the number of second data files in a second database, and the data files in the second database are obtained by transmission of the first database.
And when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to the first database, wherein the target control file is the latest first control file in the first database.
And acquiring an intermediate control file corresponding to the second database according to the target control file.
And when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to the first database and a second control file corresponding to the second database.
And acquiring an intermediate control file corresponding to the second database according to the first log file corresponding to the first database and the second control file corresponding to the second database.
And acquiring a target data file set corresponding to the second database according to the intermediate control file, wherein the number of data files in the target data file set is consistent with the first data volume.
The invention also protects a data file breakpoint continuous transmission device, which comprises:
the data volume acquisition module is used for acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database;
the target control file module is used for acquiring a target control file corresponding to the first database when the first data volume is consistent with the second data volume;
the first file acquisition module is used for acquiring an intermediate control file corresponding to a second database according to the target control file;
the non-data file acquisition module is used for acquiring a first log file corresponding to the first database and a second control file corresponding to the second database when the first data volume is inconsistent with the second data volume;
the second file acquisition module is used for acquiring an intermediate control file corresponding to the second database according to the first log file corresponding to the first database and the second control file corresponding to the second database;
and the target data file set acquisition module is used for acquiring a target data file set corresponding to the second database according to the intermediate control file, and the number of data files in the target data file set is consistent with the first data volume.
The invention protects a computer readable storage medium, which stores a computer program, and the computer program realizes the data file breakpoint continuous transmission method when being executed by a processor.
Compared with the prior art, the invention has obvious advantages and beneficial effects. By the technical scheme, the data file breakpoint continuous transmission method, the device, the computer equipment and the storage medium provided by the invention can achieve considerable technical progress and practicability, have industrial wide utilization value and at least have the following advantages:
the invention discloses a method, a device, equipment and a storage medium for breakpoint continuous transmission of a data file, wherein the method comprises the following steps: acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database, and the data files in the second database are obtained by transmitting the first database; when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to a first database, wherein the target control file is the latest first control file in the first database; acquiring an intermediate control file corresponding to a second database according to the target control file; when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to a first database and a second control file corresponding to a second database; acquiring an intermediate control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database; acquiring a target data file set corresponding to the second database according to the intermediate control file, wherein the number of data files in the target data file set is consistent with the first data amount; therefore, the problem that updating of the data files in the databases cannot be asynchronous due to the fact that the safety and the stability of the data files are guaranteed and further the working efficiency is affected can be avoided.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more clearly understood, the following preferred embodiments are described in detail with reference to the accompanying drawings.
Drawings
Fig. 1 is a flowchart of a method for breakpoint resuming of a data file according to an embodiment of the present invention;
fig. 2 is a flowchart of step S500 according to an embodiment of the present invention;
fig. 3 is a flowchart of step S501 according to an embodiment of the present invention;
fig. 4 is a flowchart of step S600 according to an embodiment of the present invention;
fig. 5 is a flowchart of step S601 according to an embodiment of the present invention;
fig. 6 is a flowchart of step S603 according to a first embodiment of the present invention;
FIG. 7 is a schematic structural diagram of an apparatus for generating a user representation according to a second embodiment of the present invention;
fig. 8 is a schematic structural diagram of a module 50 according to a second embodiment of the present invention;
fig. 9 is a schematic structural diagram of a module 51 according to a second embodiment of the present invention;
fig. 10 is a schematic structural diagram of a module 60 according to a second embodiment of the present invention;
fig. 11 is a schematic structural diagram of a module 61 according to a second embodiment of the present invention;
fig. 12 is a schematic structural diagram of a module 63 according to a second embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description will be given with reference to the accompanying drawings and preferred embodiments of a data processing system for acquiring a target position and its effects.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
As shown in fig. 1, the first embodiment provides a method for resuming a data file at a breakpoint, where the method includes the following steps:
s100, a first data volume and a second data volume are obtained, wherein the first data volume is the number of first data files in a first database, the second data volume is the number of second data files in a second database, and the data files in the second database are obtained by transmitting the first database.
Illustratively, the first database and the second database are both Oracle databases, and those skilled in the art will appreciate that the first database and the second database may also be other databases commonly used in the art, such as MySQL, etc.
Specifically, the first data file is a data file stored in a first database.
Specifically, the second data file is a data file stored in a second database.
S200, when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to the first database.
Specifically, the first database includes a plurality of first control files, a plurality of first data files, and a plurality of first log files.
Specifically, the target control file is a first control file that is the latest in the first database. Further, the first control file includes a first data file ID and file information corresponding to the first data file ID, and may be understood as: and the file information corresponding to the first data file ID is metadata corresponding to the first data file.
S300, acquiring an intermediate control file corresponding to the second database according to the target control file.
Specifically, the intermediate control file corresponding to the second database is a control file obtained by updating a second control file corresponding to the second database.
Further, the second control file includes a second data file ID and file information corresponding to the second data file ID, which can be understood as: and the file information corresponding to the second data file ID is metadata corresponding to the second data file.
S400, when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to the first database and a second control file corresponding to the second database.
S500, acquiring a middle control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database.
As shown in fig. 2, S500 specifically includes the following steps:
s501, acquiring a first target similarity corresponding to a second database according to the first log file and the second control file, wherein the first target similarity is represented by the similarity between the first log file and the second control file.
As shown in fig. 3, S501 specifically includes the following steps:
s5011, obtaining a feature vector set A = { A } corresponding to the first log file 1 ，A 2 ，……，A i ，……，A m }，A i I =1,2, \ 8230; \ 8230;, m, m being the number of first intermediate data files, for the first eigenvector corresponding to the ith first intermediate data file ID in the first database.
Specifically, the first intermediate data file ID is a unique identifier of a first intermediate data file, where the first intermediate data file is a first data file recorded by a first log file.
S5013, acquiring the second control textFeatures corresponding to the elements vector set B = { B = 1 ，B 2 ，……，B j ，……，B n }，B j J =1,2, \ 8230; \8230for the second eigenvector corresponding to the jth second intermediate data file ID, n, n being the number of second intermediate data files.
Specifically, the second intermediate data file ID is a unique identification of the second intermediate data file, where the second intermediate data file is consistent with the second data file.
S5015 according to A i And B j Obtaining B j Corresponding first degree of similarity F i j (ii) a Those skilled in the art know that the method for obtaining the similarity between any two vectors in the prior art is not described herein again.
S5017, when F i j ＞F 0 When it is, B j A corresponding second data file as a first designated data file, wherein F 0 Is a preset first target similarity threshold.
S5019, acquiring a first target similarity F corresponding to the second database according to the number of the first designated data files, wherein F meets the following conditions:
f = z/(m + n), where z is the number of first specified data files.
S503, when the first target similarity is within a preset first similarity threshold interval, acquiring an intermediate control file corresponding to a second database.
And S505, when the first target similarity is not within a preset first similarity threshold interval, acquiring the target control file.
And S507, acquiring an intermediate control file corresponding to the second database according to the target control file.
Specifically, the first similarity threshold interval H = { H 1 ，H 2 In which H 1 A similarity threshold upper limit value of H, H 2 The similarity threshold lower limit value is H.
Preferably, H 1 The following conditions are met:
H 1 =（n-z）/（m+n）。
preferably, H 2 The following conditions are met:
H 2 =（m-z）/（m+n）。
by the method, the mode of updating the data file information in the control file can be determined according to the control file and the log in the two databases, so that on one hand, the problem of low efficiency caused by repeated updating of the same data file information is avoided; on the other hand, the situation that the total data file information of the control files cannot be completely updated is avoided, and the accuracy of updating the control files in the database for receiving the transmission data files is improved.
S600, acquiring a target data file set corresponding to a second database according to the intermediate control file, wherein in the second database, data files in the first database are subjected to breakpoint continuous transmission to the second database, all the acquired data files constitute the target data file set, and the number of the data files in the target data file set is consistent with that of the data files in the first database.
As shown in fig. 4, S600 specifically includes the following steps:
s601, obtaining a key data file corresponding to the second database according to the intermediate control file.
As shown in fig. 5, S601 specifically includes the following steps:
s6011, obtaining a feature vector set C = { C) corresponding to the second database 1 ，C 2 ，……，C j ，……，C n }，C j J =1,2, \ 8230; \8230; n, n is the number of second data files for the third eigenvector corresponding to the jth second data file ID.
S6012, obtaining a feature vector set D = { D) corresponding to the intermediate control file 1 ，D 2 ，……，D r ，……，D s }，D r For the first eigenvector corresponding to the r-th third intermediate data file ID, r =1,2, \8230, s is the number of third intermediate data files.
Specifically, the third intermediate data file ID is a unique identification of the third intermediate data file ID, where the third intermediate data file is a data file recorded by the intermediate control file.
S6013, according to C j And D r Obtaining D r Corresponding second degree of similarity E j r (ii) a Those skilled in the art know that the method for obtaining the similarity between any two vectors in the prior art is not described herein again.
S6014, when E j r ＞E 0 When it is, D r A corresponding third intermediate data file as a second designated data file, wherein E 0 Is a preset second target similarity threshold.
S6015, according to the second designated data file, a second target similarity E corresponding to a second database is obtained, wherein E meets the following conditions:
e = g/(s + n), where g is the number of second specified data files.
S6016, when the E is in the U, transmitting the first log file in the first database to a second database, wherein the U is a preset second similarity threshold interval.
Further, the air conditioner is provided with a fan, U = { U = { (U) 1 ，U 2 In which U 1 Is the upper limit value of the similarity threshold value of U 2 Is the lower limit value of the similarity threshold of U.
Preferably, U 1 The following conditions are met:
H 1 =（n-g）/（s+n）。
preferably, U 2 The following conditions are met:
U 2 =（s-g）/（s+n）。
s6017, when the E is not in the U, acquiring other third intermediate data files except the second specified data file in the D from the first database as key data files.
By the aid of the method and the device, transmission of the data files which are not received in the database for receiving the transmission data files can be determined to be recovered according to the difference between the data files in the two databases, and safety and stability of the data files are improved.
And S603, acquiring a key log file corresponding to the second database according to the key data file corresponding to the second database.
As shown in fig. 6, S603 specifically includes the following steps:
s6031, when the E is in the U, taking a target log file as a key log file, wherein the target log file is a latest first log file in the first database;
and S6033, when the E is not in the U, taking the day file corresponding to the key data file as a key log file.
S605, transmitting the key log file corresponding to the second database, and acquiring a target data file set corresponding to the second database, which can be understood as: according to the key log file corresponding to the second database, performing recovery processing on the data file in the second database to obtain a target data file set corresponding to the second database.
The embodiment provides a method for breakpoint resuming of a data file, where the method includes: acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database; when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to a first database; acquiring an intermediate control file corresponding to a second database according to the target control file; when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to a first database and a second control file corresponding to a second database; acquiring a middle control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database; acquiring a target data file set corresponding to a second database according to the intermediate control file; therefore, the problem that the safety and the stability of the data files are influenced and further the working efficiency is influenced due to the fact that the data files in the databases cannot be updated synchronously can be avoided.
Example two
The second embodiment provides a data file breakpoint resuming device, as shown in fig. 7, the device includes:
the data volume obtaining module 10 is configured to obtain a first data volume and a second data volume, where the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database, where the data files in the second database are obtained by transmitting from the first database.
Illustratively, the first database and the second database are both Oracle databases, and those skilled in the art will appreciate that the first database and the second database may also be other databases commonly used in the art, such as MySQL, etc.
Specifically, the first data file is a data file stored in a first database.
Specifically, the second data file is a data file stored in a second database.
And the target control file module 20 is configured to, when the first data volume is consistent with the second data volume, obtain a target control file corresponding to the first database.
Specifically, the first database includes a plurality of first control files, a plurality of first data files, and a plurality of first log files.
Specifically, the target control file is a first control file which is latest in a first database; it can be understood that: in the first database, the time point corresponding to the target control file is larger than any one of the first control files except the target control file.
Further, the first control file includes a first data file ID and file information corresponding to the first data file ID, and may be understood as: and the file information corresponding to the first data file ID is metadata corresponding to the first data file.
A first file obtaining module 30, where the first file obtaining module 30 is configured to obtain an intermediate control file corresponding to the second database according to the target control file.
Specifically, the intermediate control file corresponding to the second database is a control file obtained by updating a second control file corresponding to the second database.
Further, the second control file includes a second data file ID and file information corresponding to the second data file ID, which can be understood as: and the file information corresponding to the second data file ID is metadata corresponding to the second data file.
A non-data file obtaining module 40, where the non-data file obtaining module 40 is configured to obtain a first log file corresponding to the first database and a second control file corresponding to the second database when the first data volume is inconsistent with the second data volume.
Specifically, the first log file is a current log file in the first database.
And a second file obtaining module 50, where the second file obtaining module 50 is configured to obtain an intermediate control file corresponding to the second database according to the first log file corresponding to the first database and the second control file corresponding to the second database.
As shown in fig. 8, the second file acquiring module 50 further includes:
a first similarity obtaining module 51, where the first similarity obtaining module 51 is configured to obtain a first target similarity corresponding to a second database according to the first log file and the second control file, where the first target similarity represents a similarity between the first log file and the second control file.
As shown in fig. 9, the first similarity obtaining module 51 further includes:
a first feature vector set obtaining module 511, where the first feature vector set obtaining module 511 is configured to obtain a feature vector set a = { a =, = { a } corresponding to the first log file 1 ，A 2 ，……，A i ，……，A m }，A i A first characteristic corresponding to the ith first intermediate data file ID in the first databaseVector, i =1,2, \8230;, m, m is the first intermediate data file number.
Specifically, the first intermediate data file ID is a unique identifier of a first intermediate data file, where the first intermediate data file is a first data file recorded by a first log file.
A second feature vector set obtaining module 513, where the second feature vector set obtaining module 513 is configured to obtain a feature vector set B = { B } corresponding to the second control file 1 ，B 2 ，……，B j ，……，B n }，B j For the second eigenvector corresponding to the jth second intermediate data file ID, j =1,2, \8230, n is the number of second intermediate data files.
Specifically, the second intermediate data file ID is a unique identification of the second intermediate data file, where the second intermediate data file is consistent with the second data file.
A first similarity obtaining module 515, wherein the first similarity obtaining module 515 is configured to obtain the first similarity according to a i And B j Obtaining B j Corresponding first degree of similarity F i j (ii) a Those skilled in the art know that the method for obtaining the similarity between any two vectors in the prior art is adopted, and details are not repeated herein.
A first specified data file obtaining module 517, where the first specified data file obtaining module 517 is used when F is i j ＞F 0 When it is, B j A corresponding second data file as a first designated data file, wherein F 0 Is a preset first target similarity threshold.
A first target similarity obtaining module 519, where the first target similarity obtaining module 519 is configured to obtain, according to the number of the first specified data files, a first target similarity F corresponding to a second database, where F meets the following condition:
f = z/(m + n), where z is the number of first specified data files.
The first executing module 53, where the first executing module 53 is configured to obtain an intermediate control file corresponding to a second database when the first target similarity is within a preset first similarity threshold interval.
A target control file obtaining module 55, where the target control file obtaining module 55 is configured to obtain the target control file when the first target similarity is not within a preset first similarity threshold interval.
And the second execution module 57, where the second execution module 57 is configured to obtain an intermediate control file corresponding to the second database according to the target control file.
In particular, the amount of the solvent to be used, the first similarity threshold interval H = { H 1 ，H 2 In which H 1 A similarity threshold upper limit value of H, H 2 And H is the lower limit value of the similarity threshold value of H.
Preferably, H 1 The following conditions are met:
H 1 =（n-z）/（m+n）。
preferably, H 2 The following conditions are met:
H 2 =（m-z）/（m+n）。
a target data file set obtaining module 60, where the target data file set obtaining module 60 is configured to obtain a target data file set corresponding to the second database according to the intermediate control file.
As shown in fig. 10, the target data file set acquisition module 60 further includes:
and a key data file obtaining module 61, where the key data file obtaining module 61 is configured to obtain a key data file corresponding to the second database according to the intermediate control file.
As shown in fig. 11, the key data file obtaining module 61 includes:
a third feature vector set obtaining module 611, where the third feature vector set obtaining module 611 is configured to obtain a feature vector set C = { C) corresponding to the second database 1 ，C 2 ，……，C j ，……，C n }，C j J =1,2, \8230; \8230, n is the number of the second data files for the third eigenvector corresponding to the jth second data file ID.
A fourth feature vector set obtaining module 612, where the fourth feature vector set obtaining module 612 is configured to obtain a feature vector set D = { D } corresponding to the intermediate control file 1 ，D 2 ，……，D r ，……，D s }，D r For the first eigenvector corresponding to the r-th third intermediate data file ID, r =1,2, \8230, s is the number of third intermediate data files.
Specifically, the third intermediate data file ID is a unique identification of the third intermediate data file ID, where the third intermediate data file is a data file recorded by the intermediate control file.
A second similarity obtaining module 613, where the second similarity obtaining module 613 is configured to obtain the similarity according to C j And D r Obtaining D r Corresponding second degree of similarity E j r (ii) a Those skilled in the art know that the method for obtaining the similarity between any two vectors in the prior art is not described herein again.
A second designated data file acquisition module 614, wherein the second designated data file acquisition module 614 is used when E j r ＞E 0 When it is, D r The corresponding third intermediate data file serves as the second designated data file.
A second target similarity obtaining module 615, where the second target similarity obtaining module 615 is configured to obtain a second target similarity E corresponding to a second database according to the second specified data file, where E meets the following condition:
e = g/(s + n), where g is the number of second designated data files.
A fourth executing module 616, where the fourth executing module 616 is configured to transmit the first log file in the first database to the second database when E is within U, where U is a preset second similarity threshold interval.
Further, the air conditioner is provided with a fan, U = { U = { (U) 1 ，U 2 In which U 1 Is the upper limit value of the similarity threshold value of U 2 Is the lower limit value of the similarity threshold of U.
Preferably, U 1 The following conditions are met:
H 1 =（n-g）/（s+n）。
preferably, U 2 The following conditions are met:
U 2 =（s-g）/（s+n）。
a fifth executing module 617, where the fifth executing module 617 is configured to, when E is not in U, obtain, as a key data file, a third intermediate data file in D except for the second specified data file from the first database.
And a key log file obtaining module 63, where the key log file obtaining module 63 is configured to obtain a key log file corresponding to the second database according to a key data file corresponding to the second database.
As shown in fig. 12, the key log file obtaining module 63 includes:
a sixth execution module 631, the sixth execution module 631 being configured to take the first log file as a key log file when E is in U;
a seventh executing module 633, where the seventh executing module 633 is configured to take a target log file as a key log file when E is in U, where the target log file is a latest first log file in the first database.
A third executing module 65, where the third executing module 65 is configured to transmit the key log file corresponding to the second database, and obtain a target data file set corresponding to the second database, and may be understood as: according to the key log file corresponding to the second database, performing recovery processing on the data file in the second database to obtain a target data file set corresponding to the second database.
In one embodiment, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database;
when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to a first database;
acquiring an intermediate control file corresponding to a second database according to the target control file;
when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to a first database and a second control file corresponding to a second database;
acquiring a middle control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database;
and acquiring a target data file set corresponding to the second database according to the intermediate control file.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database;
when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to a first database;
acquiring an intermediate control file corresponding to a second database according to the target control file;
when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to a first database and a second control file corresponding to a second database;
acquiring an intermediate control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database;
and acquiring a target data file set corresponding to the second database according to the intermediate control file.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
Although the present invention has been described with reference to a preferred embodiment, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (12)
1. A breakpoint resuming method of a data file is characterized by comprising the following steps:
acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database, and the data files in the second database are obtained by transmitting the first database;
when the first data volume is consistent with the second data volume, acquiring a target control file corresponding to a first database, wherein the target control file is a latest first control file in the first database, the first control file comprises a first data file ID and file information corresponding to the first data file ID, and the file information corresponding to the first data file ID is metadata corresponding to the first data file;
acquiring an intermediate control file corresponding to a second database according to the target control file;
when the first data volume is inconsistent with the second data volume, acquiring a first log file corresponding to a first database and a second control file corresponding to a second database;
acquiring a middle control file corresponding to a second database according to a first log file corresponding to the first database and a second control file corresponding to the second database;
and acquiring a target data file set corresponding to the second database according to the intermediate control file, wherein the number of data files in the target data file set is consistent with the first data volume.
2. The method for breakpoint resuming of a data file according to claim 1, wherein the first database and the second database are both Oracle databases.
3. The method according to claim 1, wherein the first database includes a plurality of first control files, a plurality of first data files, and a plurality of first log files.
4. The method for breakpoint resuming of a data file according to claim 1, wherein acquiring, according to the first log file corresponding to the first database and the second control file corresponding to the second database, the intermediate control file corresponding to the second database specifically includes:
acquiring a first target similarity corresponding to a second database according to the first log file and the second control file, wherein the first target similarity is represented as the similarity between the first log file and the second control file;
when the first target similarity is within a preset first similarity threshold interval, acquiring an intermediate control file corresponding to a second database;
when the first target similarity is not within a preset first similarity threshold interval, acquiring the target control file;
and acquiring an intermediate control file corresponding to the second database according to the target control file.
5. The method for breakpoint resuming of a data file according to claim 1, wherein acquiring, according to the intermediate control file, a target data file set corresponding to a second database specifically includes:
acquiring a key data file corresponding to a second database according to the intermediate control file;
acquiring a key log file corresponding to a second database according to a key data file corresponding to the second database;
and transmitting the key log file corresponding to the second database, and acquiring a target data file set corresponding to the second database.
6. An apparatus for breakpoint resuming of a data file, the apparatus comprising:
the data volume acquisition module is used for acquiring a first data volume and a second data volume, wherein the first data volume is the number of first data files in a first database, and the second data volume is the number of second data files in a second database;
a target control file module, configured to obtain a target control file corresponding to a first database when the first data volume is consistent with the second data volume, where the target control file is a latest first control file in the first database, the first control file includes a first data file ID and file information corresponding to the first data file ID, and the file information corresponding to the first data file ID is metadata corresponding to the first data file;
the first file acquisition module is used for acquiring an intermediate control file corresponding to a second database according to the target control file;
the non-data file acquisition module is used for acquiring a first log file corresponding to the first database and a second control file corresponding to the second database when the first data volume is inconsistent with the second data volume;
the second file acquisition module is used for acquiring a middle control file corresponding to the second database according to the first log file corresponding to the first database and the second control file corresponding to the second database;
and the target data file set acquisition module is used for acquiring a target data file set corresponding to the second database according to the intermediate control file, and the number of data files in the target data file set is consistent with the first data volume.
7. The apparatus according to claim 6, wherein the first database and the second database are both Oracle databases.
8. The apparatus according to claim 6, wherein the first database comprises a plurality of first control files, a plurality of first data files, and a plurality of first log files.
9. The apparatus for breakpoint resuming of a data file according to claim 6, wherein the second file retrieving module further comprises:
a first similarity obtaining module, configured to obtain a first target similarity corresponding to a second database according to the first log file and the second control file, where the first target similarity is represented by a similarity between the first log file and the second control file;
the first execution module is used for acquiring an intermediate control file corresponding to a second database when the first target similarity is within a preset first similarity threshold interval;
the target control file acquisition module is used for acquiring the target control file when the first target similarity is not within a preset first similarity threshold interval;
and the second execution module is used for acquiring an intermediate control file corresponding to the second database according to the target control file.
10. The apparatus for breakpoint resuming of a data file according to claim 6, wherein the target data file set obtaining module includes:
the key data file acquisition module is used for acquiring a key data file corresponding to the second database according to the intermediate control file;
the key log file acquisition module is used for acquiring a key log file corresponding to the second database according to the key data file corresponding to the second database;
and the third execution module is used for transmitting the key log file corresponding to the second database.
11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data file breakpoint resuming method according to any one of claims 1 to 5 when executing the computer program.
12. A computer-readable storage medium, in which a computer program is stored, wherein the computer program, when executed by a processor, implements the method for breakpoint resuming of a data file according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058208.XA CN115129739B (en) | 2022-08-31 | 2022-08-31 | Data file breakpoint continuous transmission method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058208.XA CN115129739B (en) | 2022-08-31 | 2022-08-31 | Data file breakpoint continuous transmission method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115129739A CN115129739A (en) | 2022-09-30 |
| CN115129739B true CN115129739B (en) | 2022-11-25 |
Family
ID=83387927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211058208.XA Active CN115129739B (en) | 2022-08-31 | 2022-08-31 | Data file breakpoint continuous transmission method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115129739B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5544359A (en) * | 1993-03-30 | 1996-08-06 | Fujitsu Limited | Apparatus and method for classifying and acquiring log data by updating and storing log data |
| JPH0962696A (en) * | 1995-08-28 | 1997-03-07 | Hitachi Ltd | Data base managing system |
| JP2004062759A (en) * | 2002-07-31 | 2004-02-26 | Hitachi Ltd | Database log management method, its device and its program |
| WO2017082520A1 (en) * | 2015-11-10 | 2017-05-18 | 엘지전자 주식회사 | Database management system, and data modification and recovery method thereof |
| CN109033268A (en) * | 2018-07-10 | 2018-12-18 | 安心财产保险有限责任公司 | Method of data synchronization, device, equipment and storage medium |
| CN109597709A (en) * | 2018-11-15 | 2019-04-09 | 杭州沃趣科技股份有限公司 | A method of process breakpoint transmission is created for Oracle Dataguard |
| CN110659247A (en) * | 2018-06-13 | 2020-01-07 | 中国移动通信集团江西有限公司 | Method, device, equipment and medium for detecting continuity of call ticket file |
| CN110673987A (en) * | 2019-09-27 | 2020-01-10 | 江苏国泰新点软件有限公司 | Database recovery method, device, equipment and storage medium |
| WO2021189831A1 (en) * | 2020-03-26 | 2021-09-30 | 平安科技（深圳）有限公司 | Log optimization method, apparatus and device, and readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6389433B1 (en) * | 1999-07-16 | 2002-05-14 | Microsoft Corporation | Method and system for automatically merging files into a single instance store |
| US8145686B2 (en) * | 2005-05-06 | 2012-03-27 | Microsoft Corporation | Maintenance of link level consistency between database and file system |
| CN111241062B (en) * | 2020-01-10 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Migration method and device for database backup metadata |
-
2022
- 2022-08-31 CN CN202211058208.XA patent/CN115129739B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5544359A (en) * | 1993-03-30 | 1996-08-06 | Fujitsu Limited | Apparatus and method for classifying and acquiring log data by updating and storing log data |
| JPH0962696A (en) * | 1995-08-28 | 1997-03-07 | Hitachi Ltd | Data base managing system |
| JP2004062759A (en) * | 2002-07-31 | 2004-02-26 | Hitachi Ltd | Database log management method, its device and its program |
| WO2017082520A1 (en) * | 2015-11-10 | 2017-05-18 | 엘지전자 주식회사 | Database management system, and data modification and recovery method thereof |
| CN110659247A (en) * | 2018-06-13 | 2020-01-07 | 中国移动通信集团江西有限公司 | Method, device, equipment and medium for detecting continuity of call ticket file |
| CN109033268A (en) * | 2018-07-10 | 2018-12-18 | 安心财产保险有限责任公司 | Method of data synchronization, device, equipment and storage medium |
| CN109597709A (en) * | 2018-11-15 | 2019-04-09 | 杭州沃趣科技股份有限公司 | A method of process breakpoint transmission is created for Oracle Dataguard |
| CN110673987A (en) * | 2019-09-27 | 2020-01-10 | 江苏国泰新点软件有限公司 | Database recovery method, device, equipment and storage medium |
| WO2021189831A1 (en) * | 2020-03-26 | 2021-09-30 | 平安科技（深圳）有限公司 | Log optimization method, apparatus and device, and readable storage medium |
Non-Patent Citations (3)
| Title |
|---|
| Oracle数据库数据恢复方法的研究;徐吉等;《计算机工程》;20060105(第13期);第75-77+80页 * |
| RDBMS Based Hadoop Metadata and Log Data Management Optimization;Haiying Che等;《2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP)》;20191231;全文 * |
| 遥感数据异地备份中数据一致性方法研究与实现;程艳娜等;《遥感信息》;20110215(第01期);第37-40页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115129739A (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110750267B (en) | Service information updating method, device, computer equipment and storage medium | |
| CN109325041A (en) | Business data processing method, device, computer equipment and storage medium | |
| CN110705598A (en) | Intelligent model management method and device, computer equipment and storage medium | |
| CN107861991B (en) | Receipt data processing method and device, computer equipment and storage medium | |
| CN110704428A (en) | Data indexing method and device for block chain, computer equipment and storage medium | |
| US20130339779A1 (en) | Systematic failure remediation | |
| CN115048925A (en) | Data processing system for determining abnormal text | |
| CN110727698A (en) | Database access method and device, computer equipment and storage medium | |
| CN111241062A (en) | Migration method and device for database backup metadata | |
| CN110912908A (en) | Network protocol anomaly detection method and device, computer equipment and storage medium | |
| CN114996280B (en) | Method, device, equipment and medium for correcting field information of data table | |
| CN115129739B (en) | Data file breakpoint continuous transmission method, device, equipment and storage medium | |
| CN112541739B (en) | Method, device, equipment and medium for testing question-answer intention classification model | |
| CN110265104B (en) | Diagnostic report conformity detection method, device, computer equipment and storage medium | |
| CN114238474A (en) | Data processing method, device and equipment based on drainage system and storage medium | |
| CN109101232B (en) | Method, device, computer equipment and storage medium for product development | |
| CN115277677B (en) | Batch file hanging method and device, computer equipment and storage medium | |
| CN114968346A (en) | Method, device, equipment and storage medium for detecting field comments of DDL script | |
| CN109656948B (en) | Bitmap data processing method and device, computer equipment and storage medium | |
| JP6937146B2 (en) | Multiple learning system and multiple learning program | |
| CN113821517A (en) | Data synchronization method, device, equipment and storage medium | |
| CN108897818B (en) | Method and device for determining aging state of data processing process and readable storage medium | |
| CN112783866A (en) | Data reading method and device, computer equipment and storage medium | |
| CN111666340A (en) | Synchronous data proofreading method and device based on big data and computer equipment | |
| CN112363929A (en) | System online method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |