CN115454971A

CN115454971A - Data migration method and device, electronic equipment and storage medium

Info

Publication number: CN115454971A
Application number: CN202211189008.8A
Authority: CN
Inventors: 毛思平; 施学勤; 梁铠埼; 段晗
Original assignee: Agricultural Bank of China
Current assignee: Agricultural Bank of China
Priority date: 2022-09-28
Filing date: 2022-09-28
Publication date: 2022-12-09

Abstract

The invention discloses a data migration method, a data migration device, electronic equipment and a storage medium, wherein the method comprises the following steps: responding to a data migration instruction, acquiring data to be migrated based on a source database, and processing the data to be migrated based on preset data processing logic to obtain target migration data; storing the target migration data into a target storage space, and pushing the corresponding target migration data to a target end database for storage based on a data push instruction when the data push instruction is received; the target storage space, the source database and the target database are located in the same memory space, and the target storage space includes at least one sub-storage space. The technical scheme of the embodiment of the invention realizes that the whole data migration process is completed in the memory, does not generate any intermediate file, avoids the leakage of service data, improves the safety of data migration, and improves the data processing efficiency.

Description

Data migration method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of big data processing technologies, and in particular, to a data migration method and apparatus, an electronic device, and a storage medium.

Background

Along with the continuous enhancement of data interaction of an information system, the large-scale construction of a data warehouse and a data mart, the great use of a data analysis system, the continuous upgrading and updating of a database, the adjustment and iterative updating of an application system architecture, the great amount of data migration operation in a project production process, and the data migration synchronization process across a database platform and a cross-service architecture become more frequent.

At present, in the data migration process, data migration of different database types often requires that data of a source end database is exported into a file with a specified format, and then the file is imported into a target end database. The disadvantages of this approach are: a data file needs to be generated in the data migration process, a certain risk of data leakage exists, and problems of database process or bandwidth resource occupation and the like exist in the data file generation process.

Disclosure of Invention

The invention provides a data migration method, a data migration device, electronic equipment and a storage medium, which are used for completing the whole data migration process in a memory without generating any intermediate file, thereby avoiding leakage of service data, improving the safety of data migration and improving the data processing efficiency.

According to an aspect of the present invention, there is provided a data migration method, including:

responding to a data migration instruction, acquiring data to be migrated based on a source database, and processing the data to be migrated based on preset data processing logic to obtain target migration data;

storing the target migration data into a target storage space, and pushing corresponding target migration data to a target end database for storage based on a data pushing instruction when the data pushing instruction is received;

the target storage space, the source database and the target database are located in the same memory space, and the target storage space includes at least one sub-storage space.

According to another aspect of the present invention, there is provided a data migration apparatus, including:

the data to be migrated acquisition module is used for responding to a data migration instruction, acquiring data to be migrated based on the source database, and processing the data to be migrated based on preset data processing logic to obtain target migration data;

the data to be migrated storage module is used for storing the target migration data into a target storage space, so that when a data push instruction is received, corresponding target migration data is pushed to a target end database to be stored based on the data push instruction; the target storage space, the source database and the target database are located in the same memory space, and the target storage space includes at least one sub-storage space.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the data migration method according to any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement a data migration method according to any one of the embodiments of the present invention when the computer instructions are executed.

According to the technical scheme of the embodiment of the invention, the data to be migrated is obtained based on the source-end database in response to the data migration instruction, the data to be migrated is processed based on the preset data processing logic to obtain the target migration data, and further, the target migration data is stored in the target storage space, so that when the data push instruction is received, the corresponding target migration data is pushed to the target-end database for storage based on the data push instruction.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a data migration method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a target storage space provided according to an embodiment of the present invention;

fig. 3 is a flowchart of a data migration method according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a data migration apparatus according to a second embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device implementing the data migration method according to the embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example one

Fig. 1 is a flowchart of a data migration method according to an embodiment of the present invention, where this embodiment is applicable to a case where data stored in a source database is synchronously migrated to a target database, and the method may be executed by a data migration apparatus, where the data migration apparatus may be implemented in a form of hardware and/or software, and the data migration apparatus may be configured in a terminal and/or a server.

As shown in fig. 1, the method includes:

s110, responding to a data migration instruction, acquiring data to be migrated based on a source database, and processing the data to be migrated based on preset data processing logic to obtain target migration data.

In this embodiment, the data migration instruction may be a piece of pre-written program code, which is used to trigger data migration between different databases. In practical applications, a control for triggering data migration may be developed in advance, and when it is detected that a user triggers the control, a data migration instruction may be generated, so that when the server receives the data migration instruction, the server may respond to the data migration instruction. The source database may be a database corresponding to the data provider during the data migration process. The source database may be the source of the data migration, from which data may be derived. The data to be migrated may be data that needs to be migrated to another database. The data processing logic may be preset and is configured to process the data so that the data may meet the data processing requirements of the data type requirements of the target-side database. It should be noted that, before data migration is performed, a user may configure various data logic configuration items displayed on the display interface based on a service processing logic requirement corresponding to current data migration, and when a trigger operation of the confirmation control is detected, a complete data processing logic may be generated based on data logic configuration information displayed on the current display interface and stored, so that when data corresponding to the data processing logic is received, the data processing logic may be called, so that the called data processing logic may process the data. It should also be noted that the data processing logic may be configured based on the character set difference, field type difference and business processing logic requirement of the source end database and the target end database.

It should be noted that, in order to save database resources and storage space, data is stored in the database in the form of a data file or a database log, so that after receiving a data migration instruction, a corresponding data file or a database log may be called from the source-end database and processed, thereby obtaining data to be migrated.

Optionally, in response to the data migration instruction, obtaining data to be migrated based on the source database, including: acquiring a data file to be processed corresponding to the data migration instruction based on the source database; and analyzing the data file to be processed to obtain the data to be migrated.

In this embodiment, the data file to be processed may be a partial data file stored in the source database. It should be noted that, during the data migration process, the data file to be processed may be continuously exported from the source database until the data migration process is finished.

In practical application, when a data migration instruction is received, the data migration instruction can be responded, a to-be-processed data file corresponding to the data migration instruction is called from a source database, and further, the to-be-processed data file is analyzed, so that to-be-migrated data in a data stream form can be obtained.

It should be noted that a large number of data files are stored in the source database, and in the data migration process, the data files stored in the source database are not all exported, but a part of the data files are used as data files to be processed, so that all the data files in the source database are exported step by step, and in order to record the export process of the data files to be processed, the data migration process before the interruption can be interrupted, and when the data migration process is restarted, the data migration process before the interruption can be positioned.

Based on this, on the basis of each technical scheme, the method further comprises the following steps: and determining a file identifier of the data file to be processed, and storing the file identifier into a target configuration library.

In this embodiment, the file identifier may be an identifier for identifying a data file to be processed. It should be noted that the file identifier may be a character string generated by using a snowflake algorithm, a random algorithm, or a hash algorithm, and the character string may be composed of symbols, letters, and/or numbers. The target configuration repository may be an information repository for storing various pieces of configuration information associated with data migration.

In practical application, while the data file to be processed is analyzed, the file identifier of the data file to be processed can be determined, and the file identifier is stored in the target configuration library, so that the data file to be processed which is currently processed is recorded.

It should be noted that the data processing logic may also be stored in the target configuration library after being generated, and the target configuration library may store a variety of data processing logics, and the corresponding data processing logics may be invoked by the data identifier corresponding to the data to be migrated.

Optionally, processing the data to be migrated based on a preset data processing logic to obtain target migration data, where the processing includes: when the data to be migrated is detected, determining a data identifier corresponding to the data to be migrated, calling corresponding data processing logic from the target configuration library based on the data identifier, and processing the data to be migrated based on the data processing logic to obtain target migration data.

The data identifier may be a data identifier for identifying data to be migrated. It should be noted that the data identifier may be a character string generated by using a snowflake algorithm, a random algorithm or a hash algorithm, and the character string may be composed of symbols, letters and/or numbers. In this embodiment, the data identifier may be matched with table information of the corresponding to-be-processed data file in the source database.

In specific implementation, after the to-be-processed data file is analyzed and the to-be-migrated data is obtained, the data identifier corresponding to the to-be-migrated data can be determined, and then the corresponding data processing logic is called from the target configuration library based on the data identifier, and the to-be-migrated data is processed based on the data processing logic, so that the target migrated data which is matched with the current service processing logic requirement and meets the data storage requirement of the target end database can be finally obtained.

And S120, storing the target migration data into a target storage space, and pushing the corresponding target migration data into a target end database for storage based on the data push instruction when the data push instruction is received.

The target storage space may be a data storage space for temporarily storing the migration data in the data migration process. The target storage space comprises at least one sub-storage space. In this embodiment, the target storage space may be a storage space of a ring structure. The advantages of such an arrangement are: the data read-write efficiency can be improved while avoiding data coverage. The target storage space may include a storage pointer and a read pointer. In practical applications, a set of data stored in or fetched from the target storage space can be represented by adding one to the storage pointer and adding one to the reading pointer. Illustratively, the target storage space may be a Ring Buffer (Buffer Ring), as shown in fig. 2. It should be noted that, before the data migration process starts, the data amount of the migration data related to the data migration process may be estimated, and then, a storage space with a corresponding size is created in the memory, and the number of the required sub-storage spaces is set, so that the memory of each sub-storage space may be determined according to the total memory of the total storage space and the number of the sub-storage spaces, and each sub-storage space is connected together in the form of a ring queue, so as to finally obtain the target storage space, and meanwhile, the configuration information of the target storage space may be stored in the target configuration library, so that when the data migration operation is triggered, the corresponding target storage space may be generated based on the configuration information stored in the target configuration library.

In this embodiment, the data push instruction may be a pre-written piece of program code, and the program code may be used to trigger the data push operation. The target-side database may be a database corresponding to the data import terminal in the data migration process, and is a destination of the data migration. It should be noted that the source-side database and the target-side database may be the same type of database or different types of databases, and this is not specifically limited in this embodiment of the present invention.

The target storage space, the source database and the target database are located in the same memory space. The benefit of this arrangement is: the whole data migration process is completed in the memory, the data migration process does not fall to the ground, no intermediate file in any form is generated, the leakage of service data is physically avoided, and the safety of data migration is improved.

In practical applications, because the target storage space includes a plurality of sub storage spaces, and the data amount of the target migration data that needs to be stored each time is different, when the target migration data is stored in the target storage space, the target migration data can be stored based on the data amount of the target migration data that needs to be stored this time and the remaining memory of the sub storage spaces.

Optionally, storing the target migration data in the target storage space includes: determining the data volume of the target migration data, and comparing the data volume with the usable memory of each sub-storage space in the target storage space; determining a sub-storage space corresponding to the target migration data based on the comparison result, and storing the sub-storage space; and determining the position sequence number of the sub-storage space so as to update the storage pointer address of the target storage space based on the position sequence number.

In this embodiment, the location sequence number may be a sequence number randomly generated based on the number of each sub-storage space when the target storage space is configured. The storage pointer address may be an address in the target storage space for each target migration data.

In practical application, when storing target migration data into a target storage space, a data size of the target migration data that needs to be stored at this time may be determined first, and the data size is compared with a usable memory of each sub-storage space, when it is detected that the usable memory of any sub-storage space is not less than the data size, the sub-storage space may be used as a sub-storage space of corresponding target migration data, and the target migration data is stored into the corresponding sub-storage space, and further, a location number of the sub-storage space in the target storage space is determined, so that a storage pointer address of the target storage space may be updated based on the location number.

Further, when receiving a data push instruction, based on the data push instruction, pushing corresponding target migration data to a target end database for storage, including: when a data push instruction is detected, determining a read pointer address of a target storage space; determining a position sequence number of a sub storage space to be pushed based on the reading pointer address; and acquiring target migration data based on the position serial number and storing the target migration data in a target end database.

In this embodiment, the read pointer address may be a memory address pointed to by a read pointer in the target storage space.

In practical application, when a data push instruction is detected, a read pointer address of a target storage space can be determined and read, and then a position serial number of a sub-storage space to be pushed can be determined, and further, target migration data stored in the sub-storage space to be pushed are extracted and pushed to a target end database for storage.

It should be noted that, when the data storage and data reading process is implemented based on the target storage space, the data storage and data reading process may be implemented by a producer process and a consumer process, that is, data writing is implemented by the producer process, and data reading is implemented by the consumer process.

It should be further noted that, during the data migration process, a problem that the target-side database fails to cause interruption of the data migration process may occur, and in order to prevent the target-side database from being relocated to the process before the interruption when restarting after the interruption, the read pointer address corresponding to the target migration data may be recorded when the target migration data is pushed to the target-side database, so that when a problem occurs, the memory address during the previous data reading may be located in time.

Based on this, on the basis of each technical scheme, the method further comprises the following steps: and when the target migration data is detected to be stored in the target end database, acquiring a read pointer address corresponding to the target migration data, and storing the read pointer address into the target configuration database.

In practical applications, when it is detected that target migration data is read from a corresponding sub storage space and pushed to a target database, a read pointer address corresponding to the target migration data may be determined and stored in a target configuration library.

For example, as shown in fig. 2, the storage and reading processes of the target migration data are described by taking a ring buffer with a ring structure as the target storage space. When target migration data is received, the data size of the target migration data can be determined, meanwhile, each sub storage space in the target storage space where the data can be written is determined, the data size of the target migration data is compared with the usable memory of each sub storage space where the data can be written, the sub storage space which accords with the target migration data is determined, the target migration data is stored into the sub storage space, and meanwhile, the position of a storage pointer can be changed through a producer process, so that the address of the storage pointer is updated; when data pushing is carried out, when a data pushing instruction is received, a reading pointer address in a target storage space can be determined, then the position of a sub storage space which needs to be read at present can be determined based on the reading pointer address, and target migration data in the sub storage space is read out and pushed to a target end database.

It should be noted that, during the data migration process, the benefits of using the target storage space for data storage and reading are as follows: the data stored in the target storage space cannot be deleted unless new data are covered; when data reading is carried out, all users can read the data according to the position serial numbers, and the data reading and writing efficiency is improved.

Illustratively, as shown in fig. 3, specific execution steps of the data migration process can be specifically described. 1. Acquiring a data file to be processed from a source database; 2. performing data analysis on the data file to be processed to obtain data to be migrated in a data stream form, and meanwhile, storing a file identifier corresponding to the data file to be processed into a target configuration library; 3. calling data processing logic from the target configuration library, and processing the data to be migrated based on the data processing logic to obtain target migration data; 4. storing the target migration data into a target storage space; 5. when a data push instruction is detected, corresponding target migration data can be read from the target storage space and pushed to the target end database, and meanwhile, a read pointer address of the pushed target migration data is stored in the target configuration database.

According to the technical scheme, the data to be migrated is obtained based on the source end database in response to the data migration instruction, the data to be migrated is processed based on the preset data processing logic, the target migration data is obtained, further, the target migration data is stored in the target storage space, and when the data push instruction is received, the corresponding target migration data is pushed to the target end database for storage based on the data push instruction.

Example two

Fig. 4 is a schematic structural diagram of a data migration apparatus according to a second embodiment of the present invention. As shown in fig. 4, the apparatus includes: a to-be-migrated data acquisition module 210 and a to-be-migrated data storage module 220.

The to-be-migrated data acquiring module 210 is configured to respond to a data migration instruction, acquire, based on a source database, data to be migrated, and process, based on preset data processing logic, the data to be migrated to obtain target migration data;

the to-be-migrated data storage module 220 is configured to store the target migration data in a target storage space, so that when a data push instruction is received, based on the data push instruction, corresponding target migration data is pushed to a target-side database for storage; the target storage space, the source database and the target database are located in the same memory space, and the target storage space includes at least one sub-storage space.

Optionally, the to-be-migrated data obtaining module 210 includes: the device comprises a log acquisition unit and a log analysis unit.

The log obtaining unit is used for obtaining a data file to be processed corresponding to the data migration instruction based on the source database;

and the log analysis unit is used for analyzing the data file to be processed to obtain the data to be migrated.

Optionally, the apparatus further comprises: and the log marks the storage module.

And the log identifier storage module is used for determining the file identifier of the data file to be processed and storing the file identifier into a target configuration library.

Optionally, the to-be-migrated data obtaining module 210 further includes: and the data processing unit to be migrated.

And the data to be migrated processing unit is used for determining a data identifier corresponding to the data to be migrated when the data to be migrated is detected, calling corresponding data processing logic from a target configuration library based on the data identifier, and processing the data to be migrated based on the data processing logic to obtain the target migration data.

Optionally, the to-be-migrated data storage module 220 includes: the device comprises a data volume determining unit, a sub-storage space determining unit and a storage pointer address updating unit.

A data volume determining unit, configured to determine a data volume of the target migration data, and compare the data volume with a usable memory of each sub-storage space in the target storage space;

a sub-storage space determining unit, configured to determine and store a sub-storage space corresponding to the target migration data based on the comparison result;

and the storage pointer address updating unit is used for determining the position sequence number of the sub storage space so as to update the storage pointer address of the target storage space based on the position sequence number.

Optionally, the to-be-migrated data storage module 220 further includes: the device comprises a reading pointer address determining unit, a position sequence number determining unit and a target migration data acquiring unit.

A read pointer address determination unit, configured to determine a read pointer address of the target storage space when the data push instruction is detected;

the position sequence number determining unit is used for determining the position sequence number of the sub-storage space to be pushed based on the reading pointer address;

and the target migration data acquisition unit is used for acquiring the target migration data based on the position serial number and storing the target migration data into the target end database.

Optionally, the apparatus further comprises: and reading a pointer address storage module.

And the read pointer address storage module is used for acquiring a read pointer address corresponding to the target migration data when the target migration data is detected to be stored in the target end database, and storing the read pointer address into a target configuration library.

The data migration device provided by the embodiment of the invention can execute the data migration method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

EXAMPLE III

FIG. 5 illustrates a schematic diagram of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as the data migration method.

In some embodiments, the data migration method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data migration method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data migration method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of data migration, comprising:

2. The method of claim 1, wherein the obtaining data to be migrated based on a source database in response to the data migration instruction comprises:

acquiring a data file to be processed corresponding to the data migration instruction based on the source database;

and analyzing the data file to be processed to obtain the data to be migrated.

3. The method of claim 2, further comprising:

and determining the file identification of the data file to be processed, and storing the file identification into a target configuration library.

4. The method according to claim 1, wherein the processing the data to be migrated based on preset data processing logic to obtain target migration data comprises:

when the data to be migrated is detected, determining a data identifier corresponding to the data to be migrated, calling corresponding data processing logic from a target configuration library based on the data identifier, and processing the data to be migrated based on the data processing logic to obtain the target migration data.

5. The method of claim 1, wherein storing the target migration data into a target storage space comprises:

determining the data volume of the target migration data, and comparing the data volume with the usable memories of all the sub storage spaces in the target storage space;

determining a sub-storage space corresponding to the target migration data based on the comparison result, and storing the sub-storage space;

and determining the position sequence number of the sub storage space so as to update the storage pointer address of the target storage space based on the position sequence number.

6. The method of claim 1, wherein when receiving a data push instruction, pushing corresponding target migration data into a target-side database for storage based on the data push instruction, comprises:

when the data push instruction is detected, determining a read pointer address of the target storage space;

determining the position sequence number of the sub-storage space to be pushed based on the read pointer address;

and acquiring the target migration data based on the position sequence number and storing the target migration data in the target end database.

7. The method of claim 6, further comprising:

and when the target migration data is detected to be stored in the target end database, acquiring a read pointer address corresponding to the target migration data, and storing the read pointer address into a target configuration library.

8. A data migration apparatus, comprising:

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the data migration method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon computer instructions for causing a processor, when executed, to implement the data migration method of any one of claims 1-7.