CN114064359A - Cross-platform multi-machine-room distributed database backup system - Google Patents
Cross-platform multi-machine-room distributed database backup system Download PDFInfo
- Publication number
- CN114064359A CN114064359A CN202111341828.XA CN202111341828A CN114064359A CN 114064359 A CN114064359 A CN 114064359A CN 202111341828 A CN202111341828 A CN 202111341828A CN 114064359 A CN114064359 A CN 114064359A
- Authority
- CN
- China
- Prior art keywords
- data
- backed
- module
- sending
- delta
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a cross-platform multi-machine room distributed database backup system, which comprises: the acquisition module is used for acquiring data and generating data to be backed up; the acquisition module is used for acquiring the data characteristics of the data to be backed up; the judging module is a judging unit used for judging whether the database backup data needs to be backed up or not; the determining module is a determining unit used for secondarily judging whether the backup data of the database needs to be backed up; the sending module is used for sending the data to be backed up to a database if the data to be backed up is judged to be backed up; the database is used for receiving and storing the data to be backed up transmitted by the transmitting module; the invention can improve the efficiency of the backup data, improve the accuracy of the sending module for sending the data to be backed up, effectively save the transmission time of the backup data and improve the transmission safety of the backup data.
Description
Technical Field
The invention relates to the technical field of database backup, in particular to a cross-platform multi-machine-room distributed database backup system.
Background
Data loss is a nightmare for large and small enterprises, business data is dependent on the daily business operation lip tooth of the enterprises, and the loss of the data, even if temporary, causes irreparable loss to the enterprises. Database data is the core of a system application.
In the prior art, when data is backed up, the data which is not backed up cannot be classified according to the degree of the data to be backed up, and the data to be backed up is integrally transmitted in the data backup process, so that the problems of low data transmission efficiency and unsafe data transmission are caused.
Disclosure of Invention
Therefore, the invention provides a cross-platform multi-machine-room distributed database backup system which is used for solving the problems that the data to be backed up cannot be classified according to the degree of the data to be backed up, the data transmission efficiency is low and the data transmission is unsafe in the prior art.
In order to achieve the above object, the present invention provides a cross-platform multi-machine-room distributed database backup system, which is characterized by comprising:
the acquisition module comprises a plurality of acquisition units for acquiring data and generating data to be backed up;
the acquisition module is connected with the acquisition module and comprises a plurality of acquisition units for acquiring the data characteristics of the data to be backed up;
the judging module is connected with the acquisition module and comprises a plurality of judging units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the acquisition module and judging whether the backup data of the database is needed to be backed up according to the calculation result; when the judging module calculates the importance degree of the data to be backed up, the judging module compares the actual score of the data to be backed up with a preset value and judges whether the data to be backed up needs to be backed up according to the comparison result, the judging module judges that the actual score is smaller than the preset value, the judging module judges that the backup data does not need to be backed up, the judging module judges that the actual score is the preset value, the judging module sends the backup data to a determining module for secondary judgment, the judging module judges that the actual score is larger than the preset value, and the judging module judges that the backup data needs to be backed up;
the determining module is connected with the judging module and comprises a plurality of determining units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module and secondarily judging whether the backup data of the database is required to be backed up according to the calculation result; when the determining module performs secondary judgment, the determining module acquires the file name of the data to be backed up and secondarily judges whether the data to be backed up is backed up according to the overlap ratio of the file name;
the sending module is respectively connected with the judging module and the determining module and comprises a plurality of sending units which are used for sending the data to be backed up to a database for backup if the data to be backed up is judged to be needed to be backed up; when the sending module sends the data to be backed up, the sending module sends the data to be backed up by using a sending unit in an idle state;
the database is connected with the sending module and used for receiving and storing the data to be backed up transmitted by the sending module, and when the storage is finished, the database marks the data to be backed up as the backup data;
and the correcting module is connected with the sending modules and is used for respectively detecting the transmission rates of the sending modules and judging whether to correct the number of the sending units according to the detection result.
Further, when the judging module calculates the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module, the judging module calculates an actual score a of the data to be backed up, compares the actual score a with a preset score a0, and judges whether to back up the data to be backed up according to a comparison result, wherein the data characteristics include the size of the data to be backed up, the number of times of being checked, and the total time of being checked;
the preset scores A0 include a first preset score A1 and a second preset score A2, wherein A1 < A2;
when A is less than A1, the judging module judges the data to be backed up as unimportant data to be backed up without backing up the data to be backed up;
when A is not less than A1 and not more than A2, the judging module judges that the data to be backed up is the common data to be backed up, and sends the data to be backed up to the determining module for secondary judgment;
when A is larger than A2, the judging module judges that the data to be backed up is important data to be backed up and needs to be backed up.
Further, when the judging module judges that the data to be backed up is important data to be backed up, the judging module calculates a value difference value delta A, compares the value difference value delta A with a preset value difference value, and splits the data to be backed up into a plurality of parts according to a comparison result, when the splitting is completed, the judging module sends the split data to be backed up to the sending module, and the sending module uses the sending unit to transmit each split part in the data to be backed up to the database, wherein the quantity of the sending unit is the same as that of the split parts;
the preset score difference values comprise a first preset score difference value delta A1, a second preset score difference value delta A2, a third preset score difference value delta A3, a first split part quantity Q1, a second split part quantity Q2, a third split part quantity Q3 and a fourth split part quantity Q4, wherein delta A1 is smaller than delta A2 and smaller than delta A2, and Q1 is larger than or equal to 3 and smaller than Q2 and smaller than Q3 and smaller than Q4 and smaller than or equal to 20;
when the delta A is less than the delta A1, the judging module sets the splitting number of the data to be backed up to be Q1;
when the delta A1 is more than or equal to the delta A and less than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q2;
when the delta A2 is more than or equal to the delta A and less than the delta A3, the judging module sets the splitting number of the data to be backed up to be Q3;
when the delta A is equal to or more than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q4.
Further, when the determining module completes splitting the data to be backed up and the sending module uses the sending units to transmit the split parts, the correcting module detects a transmission rate Va of the sending module, compares the transmission rate Va with a corresponding preset transmission rate Vb, and determines whether the transmission rate corrects the number of the sending units according to a comparison result, where Va is V1+ V2+ V3+ -. + Vi, and Vb is V0 × (Y/Y0), where V0 is a maximum value of the transmission rate of the sending module, Vi is an actual transmission rate of each sending unit, Y0 is the number of sending units in the sending module, Y is the number of sending units that actually send the data to be backed up, and i is Y;
when Va is Vb, the correction module judges that the transmission rate meets the standard and does not need to correct the number Y of the sending units;
and when Va is less than Vb, the correction module judges that the transmission rate does not meet the standard and needs to correct the number Y of the sending units.
Further, when the correction module detects the transmission rate Va of the sending module and compares the transmission rate Va with the corresponding preset transmission rate Vb, the correction module calculates the transmission rate difference Δ Vi of each sending unit, compares the transmission rate difference Δ Vi with the maximum value Δ Vmax of the preset transmission rate difference in sequence, and judges whether the corresponding sending unit has a fault according to the comparison result, wherein the Δ Vi is Vb-Vi;
when the delta Vi is more than or equal to the delta Vmax, the correction module judges that the sending unit has a fault, and uses a free-time sending unit adjacent to the sending unit to send the data to be backed up;
when Δ Vi < Δvmax, the correction module determines that the transmission unit has not failed, and transmits data to be backed up using the transmission unit.
Further, when the correction module determines that the number Y of the transmission units needs to be corrected, the correction module records the number of the transmission units after correction as Y1, and sets Y1 to Y x (1+ (Va/Vb)), wherein when Y1 is not an integer, the correction module rounds down.
Further, when the modification module determines that the number of the sending units needs to be modified to be Y1, the modification module compares the modified number of the sending units Y1 with the number Yb of the sending units currently in an idle state;
when Y1 is greater than Yb, the correction module judges that the number of the corrected sending units does not meet the standard, and sets the number of the sending units as Yb;
and when Y1 is less than or equal to Yb, the correction module judges that the number of the corrected sending units meets the standard and sets the number of the sending units as Y1.
Further, when the judging module judges that the data to be backed up is the common data to be backed up and sends the data to be backed up to the determining module for secondary judgment, the determining module acquires the file name of the data to be backed up and compares the file name with the backup data stored in the database to judge the contact ratio of the file name, and when the contact ratio of the file name is more than or equal to 97%, the determining module judges that the data to be backed up is the important data to be backed up and sends the data to be backed up to the sending module.
Further, the data characteristics further include a backup mark, and when the judgment module calculates the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the acquisition module and the data to be backed up includes the backup mark, the judgment module judges that the data to be backed up is important data to be backed up and sends the data to be backed up to the sending module.
Further, the actual score a is calculated using equation (1):
A＝((B/B0)+(C/C0)+(D/D0))×100％ (1)；
the data backup method comprises the following steps that B is the actual size of data to be backed up, B0 is the size of preset data to be backed up, C is the number of times of actually being checked in a statistical period t, C0 is the number of times of actually being checked in the statistical period t, D is the total time of actually being checked in the statistical period t, and D0 is the total time of actually being checked in the statistical period t.
Compared with the prior art, the method has the advantages that the judging module calculates the score of the data to be backed up according to the data characteristics of the data to be backed up, compares the score of the data to be backed up with the preset score, and determines the data to be backed up which needs to be backed up.
Particularly, the invention further carries out secondary judgment on the relevance of the data to be backed up through the determining module, and backs up the data to be backed up with the relevance meeting the preset standard, so that the efficiency of backing up the data is improved, and the accuracy of sending the data to be backed up by the sending module is improved.
Particularly, the sending module of the invention is provided with a plurality of sending units, when the sending module sends the data to be backed up, the sending module sends the data to be backed up by selecting a plurality of idle sending units, and sends the data to be backed up by using the sending units, thereby effectively improving the efficiency of backing up the data.
Furthermore, the judgment module is provided with a preset score A0, when the importance degree of the data to be backed up is judged, the judging module calculates the actual value A of the data to be backed up, compares the actual value A with a preset value A0, judges whether the data to be backed up is backed up according to the comparison result, grasps the value of each data to be backed up in real time through the judging module, on one hand, by judging the importance degree of the data to be backed up, the data to be backed up with low score can be avoided from being backed up so as to improve the efficiency of data backup, on the other hand, the judgment is carried out by a score comparison mode, the scores of the data with backup are compared with the standard value one by one, so that the data to be backed up with low scores can be avoided from being backed up, the efficiency of backing up data is improved, and meanwhile, the accuracy of sending the data to be backed up by the sending module can be improved.
Furthermore, when the judgment module judges that the data to be backed up is important data to be backed up, the judgment module calculates a value difference value delta A, compares the value difference value delta A with a preset value difference value, divides the data to be backed up into a plurality of parts according to a comparison result, and transmits the divided parts through different sending units.
Further, when the sending module transmits the data to be backed up, the correcting module obtains the actual transmission rate Va of the transmission sending module, compares the transmission rate Va with the corresponding preset transmission rate Vb, and judges whether the transmission rate corrects the number of the sending units according to the comparison result, the transmission time of the data to be backed up can be accurately grasped by comparing the sending rate of the actual sending module with the corresponding preset transmission rate, and when the transmission rate does not meet the standard, the number of the sending units is corrected to enable the transmission rate of the actual sending module to meet the standard, and the number of the sending units is corrected by the correcting module, so that the efficiency of the data backup can be effectively improved.
Furthermore, a maximum transmission rate value is preset in the modification module, when the modification module performs transmission rate comparison, the difference between each sending unit and the preset value is calculated to compare the actual transmission rate difference with the preset transmission rate difference, when the actual transmission rate difference is greater than the preset transmission rate difference, the modification module determines that the sending module has a fault, and the modification module calculates the actual transmission rate difference in real time.
Furthermore, when the number of the sending units is corrected, the correction module appropriately corrects the number of the sending units according to the ratio of the actual transmission rate to the corresponding preset transmission rate, so that the number of the sending units can be corrected in real time in the transmission process, and the number of the sending units is corrected in real time through the correction module, so that the efficiency of data backup is improved more effectively.
Further, when the determining module performs secondary judgment on the data to be backed up, the determining module obtains the file names of the data to be backed up, compares the file names in the data to be backed up with the file names in the backup data stored in the database in sequence, marks the data to be backed up as important data to be backed up when the coincidence degree of the file names meets the standard, and can more accurately judge the data to be backed up through the secondary judgment of the determining module, so that the accuracy of sending the data to be backed up by the sending module is improved, and the efficiency of backing up the data is improved.
Furthermore, when the judging module calculates the score of the data to be backed up, the importance degree of the data to be backed up can be mastered more accurately by grading the data to be backed up according to the size of the data to be backed up, the number of times of being checked in the statistical period, the total time of being checked in the statistical period, multiple angles and multiple dimensions, so that the accuracy of sending the data to be backed up by the sending module can be improved, and the efficiency of backing up the data is improved.
Drawings
Fig. 1 is a schematic structural diagram of a cross-platform multi-computer-room distributed database backup system according to the present invention.
Detailed Description
In order that the objects and advantages of the invention will be more clearly understood, the invention is further described below with reference to examples; it should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and do not limit the scope of the present invention.
It should be noted that in the description of the present invention, the terms of direction or positional relationship indicated by the terms "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, which are only for convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention.
Furthermore, it should be noted that, in the description of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
Referring to fig. 1, a schematic structural diagram of the cross-platform multi-computer-room distributed database backup system according to an embodiment of the present invention includes:
the acquisition module comprises a plurality of acquisition units for acquiring data and generating data to be backed up;
the acquisition module is connected with the acquisition module and comprises a plurality of acquisition units for acquiring the data characteristics of the data to be backed up;
the judging module is connected with the acquisition module and comprises a plurality of judging units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the acquisition module and judging whether the backup data of the database is needed to be backed up according to the calculation result;
the determining module is connected with the judging module and comprises a plurality of determining units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module and secondarily judging whether the backup data of the database is needed to be backed up according to the calculation result;
the sending module is respectively connected with the judging module and the determining module and comprises a plurality of sending units which are used for sending the data to be backed up to a database for backup if the data to be backed up is judged to be needed to be backed up; when the sending module sends the data to be backed up, the sending module sends the data to be backed up by using a sending unit in an idle state;
the database is connected with the sending module and used for receiving and storing the data to be backed up transmitted by the sending module, and when the storage is finished, the database marks the data to be backed up as the backup data;
and the correcting module is connected with the sending modules and is used for respectively detecting the transmission rates of the sending modules and judging whether to correct the number of the sending units according to the detection result.
The method has the advantages that the judging module calculates the score of the data to be backed up according to the data characteristics of the data to be backed up, compares the score of the data to be backed up with the preset score and determines the data to be backed up which needs to be backed up, on one hand, the important degree of the data to be backed up is judged, so that the data to be backed up with low score can be avoided from being backed up, so that the efficiency of the data to be backed up is improved, on the other hand, the judgment is carried out in a score comparison mode, the score of each data with the backup is compared with the standard value one by one, so that the data to be backed up with low score can be avoided from being backed up, the efficiency of the data to be backed up is improved, and the accuracy of the sending module for sending the data to be backed up can also be improved.
Particularly, the invention further carries out secondary judgment on the relevance of the data to be backed up through the determining module, and backs up the data to be backed up with the relevance meeting the preset standard, so that the efficiency of backing up the data is improved, and the accuracy of sending the data to be backed up by the sending module is improved.
Particularly, the sending module of the invention is provided with a plurality of sending units, when the sending module sends the data to be backed up, the sending module sends the data to be backed up by selecting a plurality of idle sending units, and sends the data to be backed up by using the sending units, thereby effectively improving the efficiency of backing up the data.
Specifically, when the judging module calculates the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module, the judging module calculates an actual score a of the data to be backed up, compares the actual score a with a preset score a0, and judges whether to back up the data to be backed up according to a comparison result, wherein the data characteristics include the size of the data to be backed up, the number of times of being checked, and the total time of being checked;
the preset scores A0 include a first preset score A1 and a second preset score A2, wherein A1 < A2;
when A is less than A1, the judging module judges the data to be backed up as unimportant data to be backed up without backing up the data to be backed up;
when A is not less than A1 and not more than A2, the judging module judges that the data to be backed up is the common data to be backed up, and sends the data to be backed up to the determining module for secondary judgment;
when A is larger than A2, the judging module judges that the data to be backed up is important data to be backed up and needs to be backed up.
Specifically, when the judging module judges that the data to be backed up is important data to be backed up, the judging module calculates a value difference value delta A, compares the value difference value delta A with a preset value difference value, and splits the data to be backed up into a plurality of parts according to a comparison result, when the splitting is completed, the judging module sends the split data to be backed up to the sending module, and the sending module uses the sending unit to transmit each split part in the data to be backed up to the database, wherein the number of the sending unit is the same as that of the split parts;
the preset score difference values comprise a first preset score difference value delta A1, a second preset score difference value delta A2, a third preset score difference value delta A3, a first split part quantity Q1, a second split part quantity Q2, a third split part quantity Q3 and a fourth split part quantity Q4, wherein delta A1 is smaller than delta A2 and smaller than delta A2, and Q1 is larger than or equal to 3 and smaller than Q2 and smaller than Q3 and smaller than Q4 and smaller than or equal to 20;
when the delta A is less than the delta A1, the judging module sets the splitting number of the data to be backed up to be Q1;
when the delta A1 is more than or equal to the delta A and less than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q2;
when the delta A2 is more than or equal to the delta A and less than the delta A3, the judging module sets the splitting number of the data to be backed up to be Q3;
when the delta A is equal to or more than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q4.
Specifically, when the judgment module judges that the data to be backed up is important data to be backed up, the judgment module calculates a value difference value delta A, compares the value difference value delta A with a preset value difference value, splits the data to be backed up into a plurality of parts according to a comparison result, and transmits the split parts through different sending units.
Specifically, when the determining module completes splitting the data to be backed up and the sending module uses the sending units to transmit each split part, the correcting module detects a transmission rate Va of the sending module, compares the transmission rate Va with a corresponding preset transmission rate Vb, and determines whether the transmission rate corrects the number of the sending units according to a comparison result, sets Va as V1+ V2+ V3+ -. + Vi, and sets Vb as V0 × (Y/Y0), where V0 is a maximum value of the transmission rate of the sending module, Vi is an actual transmission rate of each sending unit, Y0 is the number of sending units in the sending module, Y is the number of sending units that actually send the data to be backed up, and i is Y;
when Va is Vb, the correction module judges that the transmission rate meets the standard and does not need to correct the number Y of the sending units;
and when Va is less than Vb, the correction module judges that the transmission rate does not meet the standard and needs to correct the number Y of the sending units.
Specifically, when the correction module detects the transmission rate Va of the sending module and compares the transmission rate Va with the corresponding preset transmission rate Vb, the correction module calculates the transmission rate difference Δ Vi of each sending unit, compares the transmission rate difference Δ Vi with the maximum value Δ Vmax of the preset transmission rate difference in sequence, and judges whether the corresponding sending unit has a fault according to the comparison result, wherein Δ Vi is Vb-Vi;
when the delta Vi is more than or equal to the delta Vmax, the correction module judges that the sending unit has a fault, and uses a free-time sending unit adjacent to the sending unit to send the data to be backed up;
when Δ Vi < Δvmax, the correction module determines that the transmission unit has not failed, and transmits data to be backed up using the transmission unit.
Specifically, when the sending module transmits the data to be backed up, the correcting module obtains the actual transmission rate Va of the transmission sending module, compares the transmission rate Va with the corresponding preset transmission rate Vb, and determines whether the transmission rate corrects the number of the sending units according to the comparison result, the sending rate of the actual sending module and the corresponding preset transmission rate can accurately grasp the transmission time of the data to be backed up, and when the transmission rate does not meet the standard, the number of the sending units is corrected to make the transmission rate of the actual sending module meet the standard, and the number of the sending units is corrected by the correcting module, so that the efficiency of backing up the data can be effectively improved.
Specifically, a maximum transmission rate value is also preset in the correction module, when the correction module performs transmission rate comparison, the difference between each sending unit and the preset value is calculated to compare the actual transmission rate difference with the preset transmission rate difference, when the actual transmission rate difference is greater than the preset transmission rate difference, the correction module determines that the sending module has a fault, and the correction module calculates the actual transmission rate difference in real time.
Specifically, when the correction module determines that the number Y of the transmission units needs to be corrected, the correction module records the number of the transmission units after correction as Y1, and sets Y1 to Y × (1+ (Va/Vb)), and when Y1 is not an integer, the correction module rounds down.
Specifically, when the number of the sending units is corrected, the correction module appropriately corrects the number of the sending units according to the ratio of the actual transmission rate to the corresponding preset transmission rate, so that the number of the sending units can be corrected in real time in the transmission process, and the number of the sending units is corrected in real time through the correction module, thereby more effectively improving the efficiency of data backup.
Specifically, when the modification module determines that the number of the sending units needs to be modified to Y1, the modification module compares the modified number of the sending units Y1 with the number Yb of the sending units currently in an idle state;
when Y1 is greater than Yb, the correction module judges that the number of the corrected sending units does not meet the standard, and sets the number of the sending units as Yb;
and when Y1 is less than or equal to Yb, the correction module judges that the number of the corrected sending units meets the standard and sets the number of the sending units as Y1.
Specifically, when the judging module judges that the data to be backed up is the common data to be backed up and sends the data to be backed up to the determining module for secondary judgment, the determining module acquires the file name of the data to be backed up and compares the file name with the backup data stored in the database to judge the contact ratio of the file name, and when the contact ratio of the file name is greater than or equal to 97%, the determining module judges that the data to be backed up is the important data to be backed up and sends the data to be backed up to the sending module.
Specifically, when the determining module performs secondary determination on the data to be backed up, the determining module obtains the file names of the data to be backed up, compares the file names of the data to be backed up with the file names of the backup data stored in the database in sequence, marks the data to be backed up as important data to be backed up when the coincidence degree of the file names meets the standard, and can more accurately determine the data to be backed up through the secondary determination of the determining module, so that the accuracy of sending the data to be backed up by the sending module is improved, and the efficiency of backing up the data is improved.
Specifically, the data characteristics further include a backup mark, and when the judgment module calculates the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the acquisition module and the data to be backed up includes the backup mark, the judgment module judges that the data to be backed up is important data to be backed up and sends the data to be backed up to the sending module.
Specifically, the actual score a is calculated using formula (1):
A＝((B/B0)+(C/C0)+(D/D0))×100％ (1)；
the data backup method comprises the following steps that B is the actual size of data to be backed up, B0 is the size of preset data to be backed up, C is the number of times of actually being checked in a statistical period t, C0 is the number of times of actually being checked in the statistical period t, D is the total time of actually being checked in the statistical period t, and D0 is the total time of actually being checked in the statistical period t.
Particularly, when the judging module calculates the score of the data to be backed up, the important degree of the data to be backed up can be mastered more accurately by scoring the data to be backed up in a multi-dimensional manner through the size of the data to be backed up, the number of times of being checked in the statistical period and the total time of being checked in the statistical period, so that the accuracy of sending the data to be backed up by the sending module can be improved, and the efficiency of backing up the data is improved.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention; various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A cross-platform multi-computer-room distributed database backup system is characterized by comprising:
the acquisition module comprises a plurality of acquisition units for acquiring data and generating data to be backed up;
the acquisition module is connected with the acquisition module and comprises a plurality of acquisition units for acquiring the data characteristics of the data to be backed up;
the judging module is connected with the acquisition module and comprises a plurality of judging units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the acquisition module and judging whether the backup data of the database is needed to be backed up according to the calculation result; when the judging module calculates the importance degree of the data to be backed up, the judging module compares the actual score of the data to be backed up with a preset value and judges whether the data to be backed up needs to be backed up according to the comparison result, the judging module judges that the actual score is smaller than the preset value, the judging module judges that the backup data does not need to be backed up, the judging module judges that the actual score is the preset value, the judging module sends the backup data to a determining module for secondary judgment, the judging module judges that the actual score is larger than the preset value, and the judging module judges that the backup data needs to be backed up;
the determining module is connected with the judging module and comprises a plurality of determining units which are used for calculating the importance degree of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module and secondarily judging whether the backup data of the database is required to be backed up according to the calculation result; when the determining module performs secondary judgment, the determining module acquires the file name of the data to be backed up and secondarily judges whether the data to be backed up is backed up according to the overlap ratio of the file name;
the sending module is respectively connected with the judging module and the determining module, comprises a plurality of sending units used for sending the data to be backed up to a database for backup if the data to be backed up is judged to be needed, and sends the data to be backed up by using the sending units in an idle state when the sending module sends the data to be backed up;
the database is connected with the sending module and used for receiving and storing the data to be backed up transmitted by the sending module, and when the storage is finished, the database marks the data to be backed up as the backup data;
and the correcting module is connected with the sending modules and is used for respectively detecting the transmission rates of the sending modules and judging whether to correct the number of the sending units according to the detection result.
2. The system according to claim 1, wherein when the determining module calculates the importance of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module, the determining module calculates an actual score a of the data to be backed up, compares the actual score a with a preset score a0, and determines whether to back up the data to be backed up according to the comparison result, wherein the data characteristics include the size of the data to be backed up, the number of times to be checked, and the total time to be checked;
the preset scores A0 include a first preset score A1 and a second preset score A2, wherein A1 < A2;
when A is less than A1, the judging module judges the data to be backed up as unimportant data to be backed up without backing up the data to be backed up;
when A is not less than A1 and not more than A2, the judging module judges that the data to be backed up is the common data to be backed up, and sends the data to be backed up to the determining module for secondary judgment;
when A is larger than A2, the judging module judges that the data to be backed up is important data to be backed up and needs to be backed up.
3. The system according to claim 2, wherein when the judging module judges that the data to be backed up is important data to be backed up, the judging module calculates a score difference Δ a, compares the score difference Δ a with a preset score difference, and splits the data to be backed up into a plurality of parts according to a comparison result, and when the splitting is completed, the judging module sends the split data to be backed up to the sending module, and the sending module uses the sending unit to transmit each split part of the data to be backed up to the database, wherein the sending unit and the split parts are the same in number;
the preset score difference values comprise a first preset score difference value delta A1, a second preset score difference value delta A2, a third preset score difference value delta A3, a first split part quantity Q1, a second split part quantity Q2, a third split part quantity Q3 and a fourth split part quantity Q4, wherein delta A1 is smaller than delta A2 and smaller than delta A2, and Q1 is larger than or equal to 3 and smaller than Q2 and smaller than Q3 and smaller than Q4 and smaller than or equal to 20;
when the delta A is less than the delta A1, the judging module sets the splitting number of the data to be backed up to be Q1;
when the delta A1 is more than or equal to the delta A and less than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q2;
when the delta A2 is more than or equal to the delta A and less than the delta A3, the judging module sets the splitting number of the data to be backed up to be Q3;
when the delta A is equal to or more than the delta A2, the judging module sets the splitting number of the data to be backed up to be Q4.
4. The system of claim 3, wherein when the determining module completes splitting the data to be backed up and the sending module uses the sending units to transmit the split portions, the correcting module detects a transmission rate Va of the sending module, compares the transmission rate Va with a corresponding preset transmission rate Vb, and determines whether the transmission rate corrects the number of the sending units according to a comparison result, where Va is V1+ V2+ V3+ ·+ Vi, and Vb is V0 x (Y/Y0), where V0 is a maximum value of the transmission rate of the sending module, Vi is an actual transmission rate of each sending unit, Y0 is the number of sending units in the sending module, Y is the number of sending units that actually send the data to be backed up, and i is Y;
when Va is Vb, the correction module judges that the transmission rate meets the standard and does not need to correct the number Y of the sending units;
and when Va is less than Vb, the correction module judges that the transmission rate does not meet the standard and needs to correct the number Y of the sending units.
5. The system of claim 4, wherein when the correction module detects the transmission rate Va of the sending module and compares the transmission rate Va with a corresponding preset transmission rate Vb, the correction module calculates a transmission rate difference Δ Vi of each sending unit, compares the transmission rate difference Δ Vi with a maximum value Δ Vmax of the preset transmission rate difference in sequence, and determines whether a fault exists in the corresponding sending unit according to a comparison result, wherein Δ Vi is Vb-Vi;
when the delta Vi is more than or equal to the delta Vmax, the correction module judges that the sending unit has a fault, and uses a free-time sending unit adjacent to the sending unit to send the data to be backed up;
when Δ Vi < Δvmax, the correction module determines that the transmission unit has not failed, and transmits data to be backed up using the transmission unit.
6. The system of claim 4, wherein when the modification module determines that the number of sending units Y needs to be modified, the modification module records the modified number of sending units as Y1, and sets Y1 as Y x (1+ (Va/Vb)), and wherein when Y1 is not an integer, the modification module rounds down.
7. The system of claim 6, wherein when the modification module determines that the number of sending units needs to be modified to be Y1, the modification module compares the modified number of sending units Y1 with the number Yb of sending units currently in an idle state;
when Y1 is greater than Yb, the correction module judges that the number of the corrected sending units does not meet the standard, and sets the number of the sending units as Yb;
and when Y1 is less than or equal to Yb, the correction module judges that the number of the corrected sending units meets the standard and sets the number of the sending units as Y1.
8. The system of claim 2, wherein when the determining module determines that the data to be backed up is the normal data to be backed up and sends the data to be backed up to the determining module for a second determination, the determining module obtains a filename of the data to be backed up and compares the filename with the backup data stored in the database to determine a filename overlap ratio, and when the filename overlap ratio is greater than or equal to 97%, the determining module determines that the data to be backed up is the important data to be backed up and sends the data to be backed up to the sending module.
9. The system according to claim 2 or 8, wherein the data characteristics further include backup marks, and when the determining module calculates the importance of the data to be backed up according to the data characteristics of the data to be backed up analyzed by the obtaining module and the data to be backed up includes the backup marks, the determining module determines that the data to be backed up is important data to be backed up and sends the data to be backed up to the sending module.
10. The cross-platform multi-room distributed database backup system according to claim 2, wherein the actual score a is calculated using formula (1):
A＝((B/B0)+(C/C0)+(D/D0))×100％ (1)；
the data backup method comprises the following steps that B is the actual size of data to be backed up, B0 is the size of preset data to be backed up, C is the number of times of actual viewing in a statistical period t, C0 is the number of times of preset viewing in the statistical period t, D is the total time of actual viewing in the statistical period t, and D0 is the total time of the preset viewing in the statistical period t.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341828.XA CN114064359B (en) | 2021-11-12 | 2021-11-12 | Cross-platform multi-machine-room distributed database backup system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341828.XA CN114064359B (en) | 2021-11-12 | 2021-11-12 | Cross-platform multi-machine-room distributed database backup system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114064359A true CN114064359A (en) | 2022-02-18 |
| CN114064359B CN114064359B (en) | 2023-03-31 |
Family
ID=80271907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111341828.XA Active CN114064359B (en) | 2021-11-12 | 2021-11-12 | Cross-platform multi-machine-room distributed database backup system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114064359B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016237A (en) * | 2023-03-27 | 2023-04-25 | 武汉星翊科技有限公司 | Intelligent gateway monitoring system |
| CN117149746A (en) * | 2023-10-30 | 2023-12-01 | 山东海博科技信息系统股份有限公司 | Data warehouse management system based on cloud primordial and memory calculation separation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181042A1 (en) * | 2012-12-26 | 2014-06-26 | Kabushiki Kaisha Toshiba | Information processor, distributed database system, and backup method |
| CN104182302A (en) * | 2014-09-15 | 2014-12-03 | 北京国双科技有限公司 | Database backup method and device |
| CN112800428A (en) * | 2021-01-15 | 2021-05-14 | 中国银联股份有限公司 | Method and device for judging safety state of terminal equipment |
| CN113157489A (en) * | 2021-02-24 | 2021-07-23 | 平安普惠企业管理有限公司 | Database data backup method and device and computer equipment |
| CN113360324A (en) * | 2021-08-10 | 2021-09-07 | 北京华科海讯科技有限公司 | Data backup device based on distributed file data |
| CN113467996A (en) * | 2021-07-08 | 2021-10-01 | 咪咕音乐有限公司 | Database backup method and device, computer equipment and storage medium |
-
2021
- 2021-11-12 CN CN202111341828.XA patent/CN114064359B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181042A1 (en) * | 2012-12-26 | 2014-06-26 | Kabushiki Kaisha Toshiba | Information processor, distributed database system, and backup method |
| CN104182302A (en) * | 2014-09-15 | 2014-12-03 | 北京国双科技有限公司 | Database backup method and device |
| CN112800428A (en) * | 2021-01-15 | 2021-05-14 | 中国银联股份有限公司 | Method and device for judging safety state of terminal equipment |
| CN113157489A (en) * | 2021-02-24 | 2021-07-23 | 平安普惠企业管理有限公司 | Database data backup method and device and computer equipment |
| CN113467996A (en) * | 2021-07-08 | 2021-10-01 | 咪咕音乐有限公司 | Database backup method and device, computer equipment and storage medium |
| CN113360324A (en) * | 2021-08-10 | 2021-09-07 | 北京华科海讯科技有限公司 | Data backup device based on distributed file data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016237A (en) * | 2023-03-27 | 2023-04-25 | 武汉星翊科技有限公司 | Intelligent gateway monitoring system |
| CN116016237B (en) * | 2023-03-27 | 2023-11-14 | 武汉星翊科技有限公司 | Intelligent gateway monitoring system |
| CN117149746A (en) * | 2023-10-30 | 2023-12-01 | 山东海博科技信息系统股份有限公司 | Data warehouse management system based on cloud primordial and memory calculation separation |
| CN117149746B (en) * | 2023-10-30 | 2024-02-02 | 山东海博科技信息系统股份有限公司 | Data warehouse management system based on cloud primordial and memory calculation separation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114064359B (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114064359B (en) | Cross-platform multi-machine-room distributed database backup system | |
| CN103873287B (en) | A kind of message compensation method based on ESB message monitoring platform | |
| CN106841881B (en) | Analysis method and device for electricity utilization module | |
| EP3859867B1 (en) | Battery management system, processing device, battery management method, and battery management and control system | |
| CN111204224A (en) | Power battery pack fault control system and method based on automobile | |
| CN111404356B (en) | Power supply equipment power grade correction method and corresponding correction device | |
| CN113609119A (en) | Data account checking and repairing method and device, storage medium and electronic equipment | |
| CN111949641B (en) | Method and system for cleaning and synchronizing data among multiple stages of platforms | |
| CN117668692A (en) | Intelligent electric energy monitoring method, device, equipment and medium based on fault detection | |
| CN106373028B (en) | Power maintenance system | |
| CN116668265A (en) | Method and system for monitoring I2C bus communication abnormality | |
| CN109254952B (en) | Method for verifying validity of configuration file of intelligent substation | |
| CN116068054A (en) | Battery tab quality detection system, method, equipment and storage medium | |
| CN111398675A (en) | Electric energy meter reading method, device and system | |
| CN116708488A (en) | Water conservancy Internet of things equipment state synchronization method, device, equipment and storage medium | |
| CN106547690A (en) | Based on the Software Reliability Modeling method and device under new fitting criterion | |
| CN112884353A (en) | Task reliability evaluation system | |
| JP2011192201A (en) | Remote maintenance system and remote maintenance method | |
| CN106941429B (en) | Communication reliability test method and system | |
| CN117808497B (en) | Electric power carbon emission abnormity detection module and method based on distance and direction characteristics | |
| CN114497769B (en) | Dual-redundancy lithium battery management system and method | |
| CN112622682B (en) | Electric automobile battery replacement mileage management method and system, electronic equipment and medium | |
| CN109994326B (en) | Method for balancing capacitor unit capacitor for extra-high voltage direct current transmission project | |
| EP4256354B1 (en) | Safety mechanisms for artificial intelligence units used in safety critical applications | |
| CN115839848B (en) | Data acquisition and management method for ship diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |