CN115098468A - Flexible data migration method and device, storage medium and electronic equipment - Google Patents
Flexible data migration method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN115098468A CN115098468A CN202210752254.3A CN202210752254A CN115098468A CN 115098468 A CN115098468 A CN 115098468A CN 202210752254 A CN202210752254 A CN 202210752254A CN 115098468 A CN115098468 A CN 115098468A
- Authority
- CN
- China
- Prior art keywords
- transaction data
- data
- date
- transaction
- year
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a flexible data migration method, a flexible data migration device, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a first service date year-month table corresponding to the last service date; acquiring first transaction data from a first business date year-month table; judging the data size of the first transaction data; determining second transaction data after the first natural day time in the current business date year-month table to be migrated to the migration time in the second business date year-month table corresponding to the target business date after the last business date based on the data size; migrating second transaction data in the current business date year-month table to a second business date year-month table corresponding to a target business date after the last business date based on the migration time; and receiving a data file sent by the core system after the second transaction data is subjected to the posting processing, and according to the information in the file, performing the posting processing on the second transaction data in the second business date month table to complete the joint debugging test.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a flexible data migration method and apparatus, a storage medium, and an electronic device.
Background
In a test environment, a peripheral system automatically acquires the current service date of a core system every other fixed time period and writes the current service date into a parameter file to generate a current service date year-month table; and then acquiring a current business date year-month table corresponding to the current business date and a current natural date year-month table recorded with the transaction data of all transaction dates from the parameter file. And then, the transaction data is transferred to the current business date year-month table from the current natural date year-month table every other fixed time period, then a data file sent by the core system after the transaction data is subjected to posting processing is received, and the transaction data in the current business date year-month table is subjected to warehousing processing according to information in the data file, so that the joint debugging test is completed.
The method comprises the steps that the business volume corresponding to each business date may be different, the business volume of some business dates is more, the business volume of some business dates is less, if a fixed time period is preset, transaction data are transferred to a current business date year-month table from a current natural date year-month table every other fixed time period for warehousing treatment, a situation occurs, namely when the business volume of a certain business date is less, a peripheral system finishes treating quickly, but the fixed time period is not reached, so that the transaction data in the current natural date year-month table are not transferred to the current business date-month table corresponding to the next business date, and the peripheral system cannot perform warehousing treatment on the transaction data in the next current business date-year-month table and needs to wait. However, in a test environment, the peripheral system has a limit on the warehousing processing time of transaction data, and it is necessary to increase the data warehousing processing amount of the peripheral system as much as possible within a limited time while waiting for the data warehousing processing amount which will affect the peripheral system.
Therefore, the prior art has defects and needs to be improved and developed.
Disclosure of Invention
The embodiment of the application provides a flexible data migration method and device, a storage medium and electronic equipment, which can flexibly set a migration time point of transaction data and improve the data storage processing amount of a peripheral system in a limited time.
The embodiment of the application provides a flexible data migration method, which comprises the following steps:
acquiring a first service date year-month table corresponding to a previous service date, wherein first transaction data which are migrated from a current natural date year-month table and belong to a first natural day time are recorded in the first service date year-month table, and the first natural day time corresponds to the previous service date;
acquiring the first transaction data from the first business date year-month table;
judging the data size of the first transaction data based on the acquired first transaction data;
determining to migrate second transaction data after the first natural day time in the current service date year-month table to a migration time in a second service date year-month table corresponding to a target service date after the last service date based on the data volume of the first transaction data;
migrating second transaction data in the current business date year-month table to a second business date year-month table corresponding to a target business date after the last business date based on the migration time;
and receiving a data file sent by a core system after the second transaction data is subjected to entry processing, and according to information in the file, performing entry processing on the second transaction data in the second business date month table to complete the joint debugging test.
In the flexible data migration method according to the embodiment of the present application, the determining, based on the acquired first transaction data, a data size of the first transaction data includes:
determining the transaction type and the transaction quantity included in the first transaction data based on the acquired first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
In the flexible data migration method according to the embodiment of the present application, the determining the data size of the first transaction data based on the transaction type and the transaction amount includes:
acquiring a preset transaction data warehousing processing difficulty table, wherein the processing difficulty of the transaction data corresponding to each transaction type is recorded in the transaction data warehousing processing difficulty table;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data warehousing processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
In the flexible data migration method according to the embodiment of the present application, the determining, based on the data size of the first transaction data, to migrate the second transaction data after the first natural day time in the current natural date year-month table to a migration time in a second service date year-month table corresponding to a target service date after the previous service date includes:
if the data size of the first transaction data exceeds a first preset transaction data size, determining the migration time as first preset time;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
In the flexible data migration method according to the embodiment of the present application, before migrating the second transaction data in the current natural date year-month table to the second service date year-month table corresponding to the target service date after the previous service date based on the migration time, the method further includes:
determining the second transaction data in the second service date year-month table corresponding to the target service date after the last service date, which needs to be migrated, from the current natural date year-month table;
in the flexible data migration method according to the embodiment of the present application, after the second transaction data in the second service date year-month table corresponding to the target service date after the previous service date that needs to be migrated is determined from the current natural date year-month table, the method further includes:
judging the data size of the second transaction data;
batching the second transaction data based on the data size of the second transaction data to obtain multiple batches of second transaction data;
the migrating the second transaction data to the second business date year-month table corresponding to the target business date after the last business date includes:
and sequentially transferring a plurality of batches of second transaction data to the second service date year-month table corresponding to the target service date after the last service date according to a preset sequence.
In the flexible data migration method according to the embodiment of the present application, the receiving a data file sent by the core system after the posting process is performed on the second transaction data, and according to information in the data file, performing the posting process on the second transaction data in the second business date month table to complete the joint debugging test, includes:
receiving a data file sent by a core system after the transaction data is subjected to posting processing;
acquiring data information corresponding to each batch of the second transaction data from the data file;
and on the basis of the data information, sequentially performing entry processing on each batch of second transaction data in the second business date year-month table to complete joint debugging test.
An embodiment of the present application further provides a flexible data migration apparatus, where the apparatus includes:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a first service date year-month table corresponding to a previous service date, and first transaction data which are migrated from a current natural date year-month table and belong to a first natural day and time are recorded in the first service date year-month table and correspond to the previous service date;
the second acquisition module is used for acquiring the first transaction data from the first business date year-month table;
the judging module is used for judging the data size of the first transaction data based on the acquired first transaction data;
the determining module is used for determining migration time of second transaction data after the first natural day time in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date based on the data volume of the first transaction data;
the migration module is used for migrating the second transaction data in the current natural date year-month table to a second service date year-month table corresponding to the target service date after the last service date based on the migration time;
and the processing module is used for receiving a data file sent by the core system after the second transaction data is subjected to the account entry processing, and according to the information in the file, performing the account entry processing on the second transaction data in the second business date month table to complete the joint debugging test.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program runs on a computer, the computer is enabled to execute the flexible data migration method according to any embodiment.
An embodiment of the present application further provides an electronic device, where the electronic device includes a processor and a memory, where the memory stores a computer program, and the processor calls the computer program stored in the memory to execute the flexible data migration method according to any embodiment.
According to the embodiment of the application, the second transaction data in the current business date year-month table is determined to be migrated to the migration time in the second business date year-month table corresponding to the target business date after the last business date according to the data volume of the first transaction data in the last business date year-month table. Instead of presetting a fixed time period, the time period must be satisfied every two adjacent transaction data migration time intervals. Therefore, compared with the prior art, the migration time point of the transaction data is more flexible, and the data storage processing amount of the peripheral system in limited time is increased.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can also be derived from them without inventive effort.
Fig. 1 is a schematic flowchart of a flexible data migration method according to an embodiment of the present application.
Fig. 2 is a schematic structural diagram of a flexible data migration apparatus according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of a flexible data migration apparatus according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without inventive step, are within the scope of the present application.
The embodiment of the application provides a flexible data migration method, and the flexible data migration method can be applied to electronic equipment.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a flexible data migration method according to an embodiment of the present disclosure. The flexible data migration method is applied to electronic equipment, and can comprise the following steps:
The method includes the steps of creating a new table as a new current natural day-year-month table after table data reaches a preset data threshold if the size of the table data is limited, wherein only one current natural day-year-month table is provided, and transaction data of each natural day is added to the table along with the change of the natural day.
The first transaction data includes more than one transaction data, for example, there may be 100 transaction data, 1000 transaction data, and the like.
In some embodiments, the determining, based on the acquired first transaction data, a data size of the first transaction data includes:
determining the transaction type and the transaction quantity included in the first transaction data based on the acquired first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
The transaction type, i.e., the type to which the transaction belongs, may identify a transaction type for each transaction. The transaction amount is how many transactions, for example, the first transaction data includes 100 transactions (transaction amount).
In some embodiments, said determining the data size of the first transaction data based on the transaction type and the transaction amount comprises:
acquiring a preset transaction data storage processing difficulty table, wherein the processing difficulty of the transaction data corresponding to each transaction type is recorded in the transaction data storage processing difficulty table;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data warehousing processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
The larger the processing difficulty is, the larger the transaction quantity is, and the larger the data volume of the first transaction data is; the smaller the processing difficulty and the smaller the transaction amount, the smaller the data amount of the first transaction data.
And 104, determining to migrate second transaction data after the first natural day time in the current natural date year-and-month table to migration time in a second service date year-and-month table corresponding to a target service date after the last service date based on the data size of the first transaction data.
For example, if the previous business date is 2021 year 11/month 13, the target business date after the previous business date is 2022 year 11/month 14, and the second transaction data is the transaction data corresponding to the target business date (2022 year 11/month 14).
For example, if the data size of the first transaction data is relatively small, the migration time is advanced, that is, the second transaction data after the first natural day time in the current natural date year-month table is migrated to the second service date year-month table corresponding to the target service date after the last service date in advance. If the data volume of the first transaction data is larger, the migration time is delayed backwards, namely, the second transaction data after the first natural day time in the current natural date year-month table is migrated into a second service date year-month table corresponding to the target service date after the last service date.
In some embodiments, the determining, based on the data size of the first transaction data, to migrate the second transaction data after the first natural day time in the current natural date year-month table to a migration time in a second service date year-month table corresponding to a target service date after the last service date includes:
if the data size of the first transaction data exceeds a first preset transaction data size, determining the migration time as first preset time;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
The skilled person in the art may set the first preset transaction data size and the second preset transaction data size according to the actual situation, which is not specifically limited herein. For example, the first preset transaction data size is 1000, and the second preset transaction data size is 500.
The relation among the first preset time, the second preset time and the second preset time is as follows: the first preset time is longer than the second preset time, and the second preset time is longer than the third preset time. For example, the first preset time may be 2021, 8 months, 8 pm, 5 pm, the second preset time may be 2021, 8 months, 8 pm, 4 pm, and the third preset time may be 2021, 8 months, 8 pm, 3 pm.
And 105, migrating the second transaction data in the current natural date year-month table to a second service date year-month table corresponding to the target service date after the last service date based on the migration time.
In some embodiments, the migrating the second transaction data in the current natural date year and month table to the front of the second business date year and month table corresponding to the target business date after the last business date based on the migration time further includes:
and determining the second transaction data in the second business date year-month table corresponding to the target business date after the last business date, which needs to be migrated, from the current natural date year-month table.
Since the transaction data corresponding to all the business dates are recorded in the current natural date year-month table, the transaction data corresponding to each business date needs to be determined from all the transaction data of the current natural date year-month table.
In some embodiments, after determining, from the current natural date year and month table, the second transaction data in the second service date year and month table corresponding to the target service date after the previous service date that needs to be migrated, the method further includes:
judging the data size of the second transaction data;
batching the second transaction data based on the data volume of the second transaction data to obtain a plurality of batches of second transaction data;
the migrating the second transaction data to the second business date year-month table corresponding to the target business date after the previous business date comprises:
and sequentially transferring a plurality of batches of the second transaction data to the second service date year-month table corresponding to the target service date after the last service date according to a preset sequence.
Wherein migrating transactional data in batches may improve migration efficiency. When the data amount is relatively large, the divided batches can be a little more, and when the data amount is relatively small, the divided batches can be a little less.
And 106, receiving a data file sent by the core system after the second transaction data is subjected to the entry processing, and according to the information in the file, performing the entry processing on the second transaction data in the second business date month table to complete the joint debugging test.
The data file is sent to the peripheral system by the core system after the transaction data is subjected to posting processing. The core system transmits the transaction data after posting processing to the peripheral system through a file at the latitude of the business date, and tells the peripheral system which transactions are posted (first stage). The transaction data recorded in the current natural date year-month table is the transaction data that the core system downloads to the peripheral system in the dimension of the natural date through the mq message when the transaction occurs (when the transaction is performed on the current natural date) (the second stage).
The method comprises the steps of performing posting processing on transaction data in a current business date year-month table, namely adding a plurality of pieces of posting-related information in the current business date year-month table, namely finding data which are transmitted to a peripheral system in a first stage of a core system in the current business date year-month table, and then adding related posting information (such as posting amount, posting date and posting serial number).
In some embodiments, the receiving a data file sent by the core system after the posting process is performed on the second transaction data, and according to information in the data file, the posting process is performed on the second transaction data in the second business date month table to complete the joint debugging test, where the posting process includes:
receiving a data file sent by a core system after the transaction data is subjected to posting processing;
acquiring data information corresponding to each batch of the second transaction data from the data file;
and on the basis of the data information, sequentially performing entry processing on each batch of second transaction data in the second business date year-month table to complete joint debugging test.
The first embodiment is as follows:
for example, it is understood that in the prior art, according to a fixed time, the second transaction data should be migrated to the second service date year and month table at 5 pm on 8/2021, but according to the present scheme, since it is detected that the data size of the first transaction data is smaller than or equal to the first preset transaction data size but larger than the second preset transaction data size, the migration time is advanced, and the second transaction data is migrated to the second service date year and month table at 4 pm on 8/2021. Because the data volume of the first transaction data is relatively small, the peripheral system can be judged to be capable of processing the first transaction data in the first business date year-month table quickly. For example, it is assumed that 20 minutes are required for the peripheral system to completely migrate the second transaction data into the second business date year-month table, the peripheral system can completely import the first transaction data in the first business date year-month table at 4 pm of 2021, 8, and 20 minutes, if according to the prior art, the peripheral system migrates the second transaction data into the second business date year-month table at 5 pm of 2021, 8, and 20 minutes, the peripheral system can completely import the second transaction data into the second business date year-month table at 5 pm of 2021, 8, and 20 minutes, and the peripheral system accounts for the second transaction data in the second business date year-month table from 20 minutes at 4 pm of 2021, 8, to 20 minutes at 20 pm of 2021, 8. According to the scheme, the peripheral system transfers the second transaction data to the second service date year-month table at 8 months and 8 days in 2021 and 4 pm, and then after the peripheral system processes the first transaction data in the first service date year-month table, the peripheral system also just transfers the second transaction data to the second service date year-month table completely, so that the peripheral system can directly process the second transaction data in the second service date year-month table at the moment without waiting, and the data warehousing processing amount of the peripheral system in limited time is improved.
All the above optional technical solutions may be combined arbitrarily to form optional embodiments of the present application, and are not described herein again.
In particular implementation, the present application is not limited by the execution sequence of the described steps, and some steps may be performed in other sequences or simultaneously without conflict.
As can be seen from the above, the flexible data migration method provided in the embodiment of the present application determines, according to the data size of the first transaction data in the previous service date year-month table, to migrate the second transaction data in the current service date year-month table to the migration time in the second service date year-month table corresponding to the target service date after the previous service date. Instead of presetting a fixed time period, the time period must be satisfied every two adjacent transaction data migration time intervals. Therefore, compared with the prior art, the migration time point of the transaction data is more flexible, and the data storage processing amount of the peripheral system in limited time is increased.
The embodiment of the application further provides a flexible data migration device, and the flexible data migration device can be integrated in the electronic device.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a flexible data migration apparatus according to an embodiment of the present disclosure. The flexible data migration apparatus 30 may include:
the first obtaining module 31 is configured to obtain a first service date year-month table corresponding to a previous service date, where first transaction data that is migrated from a current natural date year-month table and belongs to a first natural time is recorded in the first service date year-month table, and the first natural time corresponds to the previous service date;
a second obtaining module 32, configured to obtain the first transaction data from the first business date year-month table;
the judging module 33 is configured to judge the data size of the first transaction data based on the acquired first transaction data;
a determining module 34, configured to determine, based on the data size of the first transaction data, that second transaction data after the first natural day time in the current natural date year-month table is migrated to a migration time in a second service date year-month table corresponding to a target service date after the previous service date;
the migration module 35 is configured to migrate, based on the migration time, second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the previous service date;
and the processing module 36 is configured to receive a data file sent by the core system after the second transaction data is subjected to posting processing, and perform posting processing on the second transaction data in the second service business date month table according to information in the data file to complete the joint debugging test.
In some embodiments, the determining module 33 is configured to determine, based on the acquired first transaction data, a transaction type and a transaction quantity included in the first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
In some embodiments, the determining module 33 is configured to obtain a preset transaction data warehousing processing difficulty table, where the transaction data warehousing processing difficulty table records the processing difficulty of the transaction data corresponding to each transaction type;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data warehousing processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
In some embodiments, the determining module 34 is configured to determine the migration time as a first preset time if the data size of the first transaction data exceeds a first preset transaction data size;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
In specific implementation, the modules may be implemented as independent entities, or may be combined arbitrarily and implemented as one or several entities.
As can be seen from the above, the flexible data migration apparatus 30 provided in the embodiment of the present application acquires, by using the first acquisition module 31, the first business date year-month table corresponding to the previous business date, where the first business date-month table records the first transaction data that is migrated from the current natural date year-month table and belongs to the first natural time, and the first natural time corresponds to the previous business date; acquiring the first transaction data from the first business date year-month table through a second acquisition module 32; judging the data size of the first transaction data by a judging module 33 based on the acquired first transaction data; determining, by the determining module 34, based on the data size of the first transaction data, migration time for migrating second transaction data after the first natural day time in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the previous service date; migrating second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date through a migration module 35 based on the migration time; and receiving a data file sent by the core system after posting the second transaction data through the processing module 36, and posting the second transaction data in the second business date year-month table according to information in the file to complete the joint debugging test.
Referring to fig. 3, fig. 3 is another schematic structural diagram of a flexible data migration apparatus according to an embodiment of the present disclosure, in which the flexible data migration apparatus 30 includes a memory 120, one or more processors 180, and one or more applications, where the one or more applications are stored in the memory 120 and configured to be executed by the processors 180; the processor 180 may include a first obtaining module 31, a second obtaining module 32, a determining module 33, a determining module 34, a migrating module 35, and a processing module 36.
For example, the structures and connection relationships of the above components may be as follows:
the memory 120 may be used to store applications and data. The memory 120 stores applications containing executable code. The application programs may constitute various functional modules. The processor 180 executes various functional applications and data processing by running the application programs stored in the memory 120. Further, the memory 120 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 120 may also include a memory controller to provide the processor 180 with access to the memory 120.
The processor 180 is a control center of the device, connects various parts of the entire terminal using various interfaces and lines, performs various functions of the device and processes data by running or executing an application program stored in the memory 120 and calling data stored in the memory 120, thereby monitoring the entire device. Optionally, processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, and the like.
Specifically, in this embodiment, the processor 180 loads the executable code corresponding to the processes of one or more application programs into the memory 120 according to the following instructions, and the processor 180 runs the application programs stored in the memory 120, thereby implementing various functions:
the first obtaining module 31 is configured to obtain a first service date year-month table corresponding to a previous service date, where the first service date year-month table records first transaction data migrated from a current natural date year-month table and belonging to a first natural time, and the first natural time corresponds to the previous service date;
a second obtaining module 32, configured to obtain the first transaction data from the first business date year-month table;
the judging module 33 is configured to judge the data size of the first transaction data based on the acquired first transaction data;
a determining module 34, configured to determine, based on the data size of the first transaction data, migration time for migrating second transaction data after the first natural day time in the current natural date year-and-month table to a second service date year-and-month table corresponding to a target service date after the previous service date;
the migration module 35 is configured to migrate, based on the migration time, second transaction data in the current natural date year-and-month table to a second service date year-and-month table corresponding to a target service date after the previous service date;
and the processing module 36 is configured to receive a data file sent by the core system after the second transaction data is subjected to posting processing, and perform posting processing on the second transaction data in the second service business date month table according to information in the data file to complete the joint debugging test.
In some embodiments, the determining module 33 is configured to determine, based on the acquired first transaction data, a transaction type and a transaction quantity included in the first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
In some embodiments, the determining module 33 is configured to obtain a preset transaction data warehousing processing difficulty table, where the transaction data warehousing processing difficulty table records the processing difficulty of the transaction data corresponding to each transaction type;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data storage processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
In some embodiments, the determining module 34 is configured to determine the migration time as a first preset time if the data size of the first transaction data exceeds a first preset transaction data size;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
The embodiment of the application also provides the electronic equipment. Referring to fig. 4, fig. 4 shows a schematic structural diagram of an electronic device provided in the embodiment of the present application, where the electronic device may be used to implement the flexible data migration method provided in the foregoing embodiment.
As shown in fig. 4, the electronic device 1200 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more computer-readable storage media (only one shown), an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more processing cores (only one shown), and a power supply 190. Those skilled in the art will appreciate that the configuration of electronic device 1200 shown in fig. 4 is not intended to be limiting of electronic device 1200 and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. Wherein:
the RF circuit 110 is used for receiving and transmitting electromagnetic waves, and performs interconversion between the electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. The RF circuitry 110 may include various existing circuit elements for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, memory, and so forth. The RF circuitry 110 may communicate with various networks such as the internet, an intranet, a wireless network, or with other devices over a wireless network.
The memory 120 may be configured to store software programs and modules, such as program instructions/modules corresponding to the flexible data migration method in the foregoing embodiments, and the processor 180 executes various functional applications and data processing by operating the software programs and modules stored in the memory 120, so as to flexibly set a migration time point of transaction data and improve a data warehousing throughput of a peripheral system in a limited time. Memory 120 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 120 may further include memory located remotely from the processor 180, which may be connected to the electronic device 1200 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input unit 130 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, the input unit 130 may include a touch-sensitive surface 131 as well as other input devices 132. The touch-sensitive surface 131, also referred to as a touch display screen or a touch pad, may collect touch operations by a user on or near the touch-sensitive surface 131 (e.g., operations by a user on or near the touch-sensitive surface 131 using a finger, a stylus, or any other suitable object or attachment), and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface 131 may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 180, and can receive and execute commands sent by the processor 180. Additionally, the touch-sensitive surface 131 may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch-sensitive surface 131, the input unit 130 may also include other input devices 132. In particular, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
The display unit 140 may be used to display information input by or provided to a user and various graphical user interfaces of the electronic device 1200, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 140 may include a Display panel 141, and optionally, the Display panel 141 may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Further, touch sensitive surface 131 may overlay display panel 141, and when touch operation is detected on or near touch sensitive surface 131, the touch operation is transmitted to processor 180 to determine the type of touch event, and then processor 180 provides a corresponding visual output on display panel 141 according to the type of touch event. Although in FIG. 4, touch-sensitive surface 131 and display panel 141 are shown as two separate components to implement input and output functions, in some embodiments, touch-sensitive surface 131 may be integrated with display panel 141 to implement input and output functions.
The electronic device 1200 may also include at least one sensor 150, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 141 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 141 and/or the backlight when the electronic device 1200 is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which may be further configured in the electronic device 1200, detailed descriptions thereof are omitted.
The electronic device 1200, via the transport module 170 (e.g., Wi-Fi module), may assist the user in emailing, browsing web pages, accessing streaming media, etc., which provides the user with wireless broadband internet access. Although fig. 4 shows the transmission module 170, it is understood that it does not belong to the essential constitution of the electronic device 1200, and may be omitted entirely as needed within the scope not changing the essence of the invention.
The processor 180 is a control center of the electronic device 1200, connects various parts of the entire cellular phone using various interfaces and lines, and performs various functions of the electronic device 1200 and processes data by operating or executing software programs and/or modules stored in the memory 120 and calling data stored in the memory 120. Optionally, processor 180 may include one or more processing cores; in some embodiments, the processor 180 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 180.
Electronic device 1200 also includes a power supply 190 (e.g., a battery) to power the various components, which in some embodiments may be logically coupled to processor 180 via a power management system to manage charging, discharging, and power consumption management functions via the power management system. The power supply 190 may also include any component including one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
Although not shown, the electronic device 1200 may further include a camera (e.g., a front camera, a rear camera), a bluetooth module, and the like, which are not described in detail herein. Specifically in this embodiment, the display unit 140 of the electronic device 1200 is a touch screen display, the electronic device 1200 further comprises a memory 120, and one or more programs, wherein the one or more programs are stored in the memory 120, and the one or more programs are configured to be executed by the one or more processors 180 and comprise instructions for:
acquiring a first service date year-month table corresponding to a previous service date, wherein first transaction data which are migrated from a current natural date year-month table and belong to a first natural day time are recorded in the first service date year-month table, and the first natural day time corresponds to the previous service date;
acquiring the first transaction data from the first business date year-month table;
judging the data size of the first transaction data based on the acquired first transaction data;
determining migration time for migrating second transaction data after the first natural day time in the current natural date year-and-month table to a second service date year-and-month table corresponding to a target service date after the last service date based on the data volume of the first transaction data;
migrating second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date based on the migration time;
and receiving a data file sent by the core system after the second transaction data is subjected to posting processing, and according to information in the file, performing posting processing on the second transaction data in the second business date year-month table to complete the joint debugging test.
In some embodiments, the processor 180 is configured to determine a transaction type and a transaction amount included in the first transaction data based on the acquired first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
In some embodiments, the processor 180 is configured to obtain a preset transaction data warehousing processing difficulty table, where the transaction data warehousing processing difficulty table records processing difficulty of transaction data corresponding to each transaction type;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data warehousing processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
In some embodiments, the processor 180 is configured to determine the migration time as a first preset time if the data size of the first transaction data exceeds a first preset transaction data size;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
In some embodiments, the processor 180 is configured to determine, from the current natural date year and month table, the second transaction data in the second business date year and month table corresponding to the target business date after the last business date that needs to be migrated.
In some embodiments, the processor 180 is configured to determine a data size of the second transaction data;
batching the second transaction data based on the data volume of the second transaction data to obtain a plurality of batches of second transaction data;
and sequentially transferring a plurality of batches of second transaction data to the second service date year-month table corresponding to the target service date after the last service date according to a preset sequence.
In some embodiments, the processor 180 is configured to receive a data file sent by the core system after the transaction data is posted;
acquiring data information corresponding to each batch of the second transaction data from the data file;
and on the basis of the data information, sequentially performing entry processing on each batch of second transaction data in the second business date year-month table to complete joint debugging test.
As can be seen from the above, an embodiment of the present application provides an electronic device 1200, where the electronic device 1200 performs the following steps: acquiring a first service date year-month table corresponding to a previous service date, wherein first transaction data which are migrated from a current natural date year-month table and belong to a first natural day time are recorded in the first service date year-month table, and the first natural day time corresponds to the previous service date; acquiring the first transaction data from the first business date year-month table; judging the data size of the first transaction data based on the acquired first transaction data; determining to migrate second transaction data after the first natural day time in the current natural date year-month table to a migration time in a second service date year-month table corresponding to a target service date after the last service date based on the data volume of the first transaction data; migrating second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date based on the migration time; and receiving a data file sent by a core system after the second transaction data is subjected to entry processing, and according to information in the file, performing entry processing on the second transaction data in the second business date month table to complete the joint debugging test.
An embodiment of the present application further provides a storage medium, where a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer executes the flexible data migration method according to any of the above embodiments.
It should be noted that, for the flexible data migration method described in the present application, it may be understood by a person of ordinary skill in the art that all or part of the process of implementing the flexible data migration method described in the present application may be implemented by controlling related hardware through a computer program, where the computer program may be stored in a computer readable storage medium, for example, a memory of an electronic device, and executed by at least one processor in the electronic device, and during the execution process, the process of implementing the embodiment of the flexible data migration method may be included as described above. The storage medium may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like.
For the flexible data migration apparatus in the embodiment of the present application, each functional module may be integrated into one processing chip, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, or the like.
The flexible data migration method, the flexible data migration device, the storage medium, and the electronic device provided by the embodiments of the present application are described in detail above. The principle and the implementation of the present application are explained herein by applying specific examples, and the above description of the embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A flexible data migration method, comprising:
acquiring a first service date year-month table corresponding to a previous service date, wherein first transaction data which are migrated from a current natural date year-month table and belong to a first natural day time are recorded in the first service date year-month table, and the first natural day time corresponds to the previous service date;
acquiring the first transaction data from the first business date year-month table;
judging the data size of the first transaction data based on the acquired first transaction data;
determining migration time for migrating second transaction data after the first natural day time in the current natural date year-and-month table to a second service date year-and-month table corresponding to a target service date after the last service date based on the data volume of the first transaction data;
migrating second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date based on the migration time;
and receiving a data file sent by the core system after the second transaction data is subjected to posting processing, and according to information in the file, performing posting processing on the second transaction data in the second business date year-month table to complete the joint debugging test.
2. The flexible data migration method according to claim 1, wherein the determining the data size of the first transaction data based on the acquired first transaction data comprises:
determining the transaction type and the transaction quantity included in the first transaction data based on the acquired first transaction data;
and judging the data size of the first transaction data based on the transaction type and the transaction quantity.
3. The flexible data migration method of claim 2, wherein said determining a data volume size of said first transaction data based on said transaction type and transaction amount comprises:
acquiring a preset transaction data warehousing processing difficulty table, wherein the processing difficulty of the transaction data corresponding to each transaction type is recorded in the transaction data warehousing processing difficulty table;
determining the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data based on the transaction data warehousing processing difficulty table;
and judging the data size of the first transaction data based on the processing difficulty of the transaction data corresponding to each transaction type in the first transaction data and the transaction quantity corresponding to each transaction type.
4. The flexible data migration method according to claim 1, wherein the determining, based on the data size of the first transaction data, to migrate the second transaction data after the first natural day time in the current natural date year-and-month table to a migration time in a second service date year-and-month table corresponding to a target service date after the previous service date comprises:
if the data size of the first transaction data exceeds a first preset transaction data size, determining the migration time as first preset time;
if the data volume of the first transaction data is smaller than or equal to a first preset transaction data volume but larger than a second preset transaction data volume, determining the migration time as a second preset time;
and if the data volume of the first transaction data is smaller than or equal to a second preset transaction data volume, determining the migration time as a third preset time, wherein the first preset time is greater than the second preset time, and the second preset time is greater than the third preset time.
5. The flexible data migration method according to claim 1, wherein the migrating the second transaction data in the current natural date year-month table to the front of the second business date year-month table corresponding to the target business date after the last business date based on the migration time further comprises:
and determining the second transaction data in the second business date year-month table corresponding to the target business date after the last business date, which needs to be migrated, from the current natural date year-month table.
6. The flexible data migration method according to claim 5, wherein after determining the second transaction data in the second business date year-month table corresponding to the target business date after the previous business date that needs to be migrated from the current natural date year-month table, further comprising:
judging the data size of the second transaction data;
batching the second transaction data based on the data size of the second transaction data to obtain multiple batches of second transaction data;
the migrating the second transaction data to the second business date year-month table corresponding to the target business date after the last business date includes:
and sequentially transferring a plurality of batches of second transaction data to the second service date year-month table corresponding to the target service date after the last service date according to a preset sequence.
7. The flexible data migration method according to claim 6, wherein the receiving core system performs the posting process on the second transaction data in the second business date month table according to the information in the data file sent after performing the posting process on the second transaction data, and completes the joint debugging test, and includes:
receiving a data file sent by a core system after the transaction data is subjected to posting processing;
acquiring data information corresponding to each batch of the second transaction data from the data file;
and on the basis of the data information, sequentially performing entry processing on each batch of second transaction data in the second business date year-month table to complete joint debugging test.
8. A flexible data migration apparatus, the apparatus comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a first service date year-month table corresponding to a previous service date, and first transaction data which are migrated from a current natural date year-month table and belong to a first natural day and time are recorded in the first service date year-month table and correspond to the previous service date;
the second acquisition module is used for acquiring the first transaction data from the first business date year-month table;
the judging module is used for judging the data size of the first transaction data based on the acquired first transaction data;
the determining module is used for determining the migration time of the second transaction data after the first natural day time in the current natural date year-month table to the second business date year-month table corresponding to the target business date after the last business date based on the data size of the first transaction data;
the migration module is used for migrating second transaction data in the current natural date year-month table to a second service date year-month table corresponding to a target service date after the last service date based on the migration time;
and the processing module is used for receiving a data file sent by the core system after the second transaction data is subjected to posting processing, and according to information in the file, posting processing is performed on the second transaction data in the second business date year-month table to complete joint debugging testing.
9. A computer-readable storage medium, in which a computer program is stored which, when run on a computer, causes the computer to carry out the flexible data migration method of any one of claims 1 to 7.
10. An electronic device, characterized in that the electronic device comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the flexible data migration method according to any one of claims 1 to 7 by calling the computer program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210752254.3A CN115098468A (en) | 2022-06-28 | 2022-06-28 | Flexible data migration method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210752254.3A CN115098468A (en) | 2022-06-28 | 2022-06-28 | Flexible data migration method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115098468A true CN115098468A (en) | 2022-09-23 |
Family
ID=83294142
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210752254.3A Pending CN115098468A (en) | 2022-06-28 | 2022-06-28 | Flexible data migration method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115098468A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115858489A (en) * | 2022-11-23 | 2023-03-28 | 中电金信软件有限公司 | Transaction processing method and device based on data migration, computer equipment and medium |
-
2022
- 2022-06-28 CN CN202210752254.3A patent/CN115098468A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115858489A (en) * | 2022-11-23 | 2023-03-28 | 中电金信软件有限公司 | Transaction processing method and device based on data migration, computer equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10956280B2 (en) | Data backup method, electronic device, and storage medium | |
| CN111142711B (en) | Firmware configuration method and device, storage medium and mobile terminal | |
| CN109857403B (en) | Page updating method and device, page processing method and device | |
| CN108470571B (en) | Audio detection method and device and storage medium | |
| WO2015154670A1 (en) | Method and apparatus for invoking application programming interface | |
| US10402222B2 (en) | Task migration method and apparatus | |
| CN109753425B (en) | Popup window processing method and device | |
| EP2869604B1 (en) | Method, apparatus and device for processing a mobile terminal resource | |
| CN107734618B (en) | Application program closing method and device, storage medium and electronic equipment | |
| CN112749074B (en) | Test case recommending method and device | |
| CN115098468A (en) | Flexible data migration method and device, storage medium and electronic equipment | |
| CN111026457B (en) | Hardware configuration method and device, storage medium and terminal equipment | |
| CN107766351B (en) | File directory identification method and device | |
| CN112415367A (en) | Drive chip abnormality detection method and device, electronic equipment and readable storage medium | |
| CN115348256A (en) | Posting data downloading method and device, storage medium and electronic equipment | |
| CN106933626B (en) | Application association method and device | |
| CN108269223B (en) | Webpage graph drawing method and terminal | |
| CN111027406B (en) | Picture identification method and device, storage medium and electronic equipment | |
| CN111081198B (en) | Data control method, data control device, storage medium and terminal equipment | |
| CN114510417A (en) | Image rendering effect testing method and device, electronic equipment and storage medium | |
| CN110225106B (en) | Data transmission method, device and storage medium | |
| CN114896159A (en) | Joint debugging test method and device, storage medium and electronic equipment | |
| CN115344554A (en) | Data processing method, device, medium and electronic equipment based on test environment | |
| CN105988801B (en) | Method and device for displaying annotation information | |
| CN115344574A (en) | Flexible data processing method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |