CN115292289A - Data filtering method, device, equipment and medium based on database migration - Google Patents
Data filtering method, device, equipment and medium based on database migration Download PDFInfo
- Publication number
- CN115292289A CN115292289A CN202210959015.5A CN202210959015A CN115292289A CN 115292289 A CN115292289 A CN 115292289A CN 202210959015 A CN202210959015 A CN 202210959015A CN 115292289 A CN115292289 A CN 115292289A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- migration
- system object
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of big data, and provides a data filtering method, a device, equipment and a medium based on database migration, wherein the method comprises the following steps: responding to the system object data filtering instruction, and determining the database type of the source database; acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache; acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache; and filtering the system object data in the data to be migrated to obtain target data suitable for migration. The invention can improve the efficiency of database migration and reduce the error rate.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a data filtering method, a data filtering device, data filtering equipment and data filtering media based on database migration.
Background
After the database is installed and deployed, necessary data for normal operation of the database, such as systematic object data of system users, system tablespaces, system roles and the like in the database, can be initialized. When the database is migrated, related database objects such as non-systematic users, tablespaces, roles, tables, views and the like used by a business system are mainly migrated into a target database, and various system object data originally carried by the database system have no business value. Therefore, the default system object data of the database needs to be filtered when the database is migrated.
In the prior art, manual filtering and screening of system object data and the like are mainly realized by inquiring a complete user list and displaying the complete user list to an operator for selection.
However, the above-mentioned method adopts manual screening, which has a high requirement on the database knowledge of the operator, and the operator needs to be familiar with the background knowledge of the database, and also needs to know which users have business data to be migrated, and if there are more business users, the operator needs to distinguish and select one by one, which results in high error rate and low migration efficiency.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data filtering method, device, apparatus and medium based on database migration, which aims to solve the problems of low database migration efficiency and high error rate.
A data filtering method based on database migration comprises the following steps:
responding to the system object data filtering instruction, and determining the database type of the source database;
acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache;
acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache;
and filtering the system object data in the data to be migrated to obtain target data suitable for migration.
According to a preferred embodiment of the present invention, the determining the database type of the source database includes:
acquiring a preset parent table in the source database;
determining database version information supported by the source database according to the parent table;
and acquiring the database type from the database version information.
According to a preferred embodiment of the present invention, the obtaining system object data in the source database according to the database type includes:
determining a child table associated with the parent table;
and acquiring the system object data from the sub-table according to the database type.
According to a preferred embodiment of the present invention, before the determining the database type of the source database, the method further includes:
maintaining the parent table and the child table by initializing insert statements; or
And collecting information input on a configuration interface, and maintaining the parent table and the child table according to the collected information.
According to a preferred embodiment of the present invention, the filtering the system object data in the data to be migrated to obtain target data suitable for migration includes:
when the system object data is a system user, querying the database user table of the source database, and eliminating the system user when querying the database user table of the source database to obtain a service user as the target data; and/or
When the system object data is a system table space, inquiring a database table space table of the source database, and eliminating the system table space when inquiring the database table space table of the source database to obtain a service table space as the target data; and/or
And when the system object data is in a system role, inquiring the database role table of the source database, and eliminating the system role when inquiring the database role table of the source database to obtain a service role as the target data.
According to a preferred embodiment of the present invention, the filtering the system object data in the data to be migrated to obtain target data suitable for migration includes:
when the system object data is a system object list, querying a database object table of the source database;
acquiring a database user to which each object list to be migrated belongs in the database object table;
and excluding the system object list belonging to the system user from each object list to be migrated to obtain a service object list as the target data.
According to a preferred embodiment of the present invention, after the obtaining the target data, the method further comprises:
determining a target database;
migrating the target data from the source database to the target database.
A database migration-based data filtering apparatus, the database migration-based data filtering apparatus comprising:
the determining unit is used for responding to the system object data filtering instruction and determining the database type of the source database;
the acquisition unit is used for acquiring the system object data in the source database according to the database type and loading the system object data to a configuration cache;
the obtaining unit is further configured to obtain data to be migrated in the source database, and obtain the system object data in the configuration cache;
and the filtering unit is used for filtering the system object data in the data to be migrated so as to obtain target data suitable for migration.
A computer device, the computer device comprising:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the database migration based data filtering method.
A computer-readable storage medium having stored therein at least one instruction for execution by a processor in a computer device to implement the database migration-based data filtering method.
According to the technical scheme, the method and the device can solve the problems of high error rate and low efficiency in the database migration process.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the data filtering method based on database migration according to the present invention.
FIG. 2 is a flow chart of another preferred embodiment of the data filtering method based on database migration according to the present invention.
FIG. 3 is a functional block diagram of a data filtering apparatus based on database migration according to a preferred embodiment of the present invention.
FIG. 4 is a schematic structural diagram of a computer device for implementing the data filtering method based on database migration according to the preferred embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a flow chart of a preferred embodiment of the data filtering method based on database migration according to the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The data filtering method based on database migration is applied to one or more computer devices, where the computer device is a device capable of automatically performing numerical calculation and/or information processing according to instructions set or stored in advance, and the hardware thereof includes but is not limited to a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an Internet Protocol Television (IPTV), an intelligent wearable device, and the like.
The computer device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network servers.
The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
Among them, artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
The Network where the computer device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
The data filtering method based on database migration disclosed in this embodiment may specifically include the following steps:
and S10, responding to the system object data filtering instruction, and determining the database type of the source database.
In at least one embodiment of the present invention, the source database may be any type of database, such as a relational database, for example: the source database may be an oracle database.
In at least one embodiment of the present invention, the system object data filtering instruction may be triggered automatically periodically, or may be triggered by a relevant staff member when there is a migration requirement, which is not limited by the present invention.
In at least one embodiment of the present invention, the determining the database type of the source database includes:
acquiring a pre-configured parent table in the source database;
determining database version information supported by the source database according to the parent table;
and acquiring the database type from the database version information.
The database version information may include information such as a database type and a database version. For example: the database version information may be: database type oracle, version 11G.
S11, obtaining the system object data in the source database according to the database type, and loading the system object data to a configuration cache.
In at least one embodiment of the present invention, the acquiring the system object data in the source database according to the database type includes:
determining a child table associated with the parent table;
and acquiring the system object data from the sub-table according to the database type.
The sub-table is used for storing the system object data corresponding to each database type in the parent table.
The system object data is the non-service data carried by the source database.
The system object data may include, but is not limited to: system users, system tablespaces, system roles, and system object lists.
For example: the system user may be: a SYSTEM; the system tablespace may be: a SYSTEM; the system roles may be: and (4) DBA.
In this embodiment, one database type corresponds to a plurality of system users, a plurality of system tablespaces, and a plurality of system roles.
It can be understood that, since the system user, the system tablespace, and the system role are non-service data of the source database itself, and system data belonging to the source database itself has no service value, there is no need to perform database migration, otherwise, not only the efficiency of database migration is affected, but also data redundancy is caused.
In at least one embodiment of the present invention, before the determining the database type of the source database, the method further includes:
maintaining the parent table and the child table by initializing insert statements; or
And acquiring information input on a configuration interface, and maintaining the parent table and the child table according to the acquired information.
Wherein the insert statement is used to insert data into the parent table or the child table, such as: an insert statement in a SQL (Structured Query Language) script is initialized to insert database version information "database type oracle, version 11G" into the parent table.
The configuration interface may be an interface for interacting with a user, for example: and related operators can delete and add data on the configuration interface so as to maintain the parent table according to user requirements.
The parent table and the child table are maintained through the embodiment, so that the subsequent query can be used conveniently.
In at least one embodiment of the present invention, the configuration cache may be a temporary cache such as a memory of an associated server.
By loading the acquired system object data to the configuration cache, the data reading efficiency can be improved, and the database migration efficiency is further improved.
And S12, acquiring the data to be migrated in the source database and acquiring the system object data in the configuration cache.
In this embodiment, the data to be migrated may include all migratable data in the source database.
The embodiment reads the system object data in the configuration cache, and can ensure the efficiency of data reading.
S13, filtering the system object data in the data to be migrated to obtain target data suitable for migration.
In at least one embodiment of the present invention, the filtering the system object data in the data to be migrated to obtain target data suitable for migration includes, but is not limited to, one or more of the following combinations:
(1) And when the system object data is a system user, querying the database user table of the source database, and eliminating the system user when querying the database user table of the source database to obtain a service user as the target data.
For example: when the source database is an oracle database, the database user table may be dba _ users.
In this example, the system user behaves as a "blacklisted user" and does not need to be migrated as it is non-traffic data.
Therefore, after the system user is excluded from the database user table, the remaining service users are service-related users, and need to be migrated with the database for subsequent use.
(2) And when the system object data is a system table space, inquiring a database table space table of the source database, and eliminating the system table space when inquiring the database table space table of the source database to obtain a service table space as the target data.
For example: when the source database is an oracle database, the database table space table may be dba _ tables.
In this example, the system tablespace corresponds to a "blacklist database space," which, because it is non-business data, does not need to be migrated.
Therefore, after the system table space is excluded from the database table space table, the remaining service table space is a service-related table space and needs to be migrated with the database for subsequent use.
(3) And when the system object data is in a system role, inquiring the database role table of the source database, and eliminating the system role when inquiring the database role table of the source database to obtain a service role as the target data.
For example: when the source database is an oracle database, the database role table can be dba _ roles.
In this example, the system role is equivalent to a "blacklist database role," which, because it is non-business data, does not require migration.
Therefore, after the system role is excluded from the database roles, the remaining service roles are service-related roles and need to be migrated with the database for subsequent use.
Through the embodiment, the system object data can be automatically filtered from the data to be migrated, so that unnecessary data migration is avoided.
Referring to fig. 2, when the system object data is a system object list, the filtering the system object data in the data to be migrated to obtain target data suitable for migration includes:
s131, querying a database object table of the source database.
For example: when the source database is an oracle database, the database object table may be dba _ objects.
S132, obtaining the database user to which each object list to be migrated belongs in the database object table.
The database user to which each object list to be migrated belongs may be the system user or the service user.
S133, excluding the system object list belonging to the system user from each object list to be migrated, and obtaining a service object list as the target data.
For example: and excluding the object list with the owner as the system user from each object list to be migrated (namely excluding the system object list belonging to the system user), wherein the rest object list is the list of all objects to be migrated (namely the service object list).
In this example, the system object list is equivalent to a "blacklisted object list" that does not require migration because it is non-business data.
Therefore, after the system object list is excluded from each object list to be migrated, the remaining service object list is an object list related to a service, and needs to be migrated with the database, so as to be used later.
In at least one embodiment of the present invention, after the obtaining the target data, the method further includes:
determining a target database;
migrating the target data from the source database to the target database.
The target database may be any type of database such as a dreams database.
Through the embodiment, the system data carried by the source database can be automatically screened out in the data migration process, other data such as service data and the like are migrated to the target database, manual screening and filtering are not needed, the database migration efficiency is improved, errors caused by manual operation are effectively avoided, and the data migration accuracy is improved.
Moreover, due to the fact that automatic screening is carried out, relevant operators do not need to have relevant knowledge of the database, and meanwhile do not need to know which business data are contained in the database, operability of database migration is improved, and the threshold is lowered.
According to the technical scheme, the database type of the source database is determined in response to the system object data filtering instruction; acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache; acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache; and filtering the system object data in the data to be migrated to obtain target data suitable for migration. The invention can improve the efficiency of database migration and reduce the error rate.
FIG. 3 is a functional block diagram of the data filtering apparatus based on database migration according to the preferred embodiment of the present invention. The data filtering apparatus 11 based on database migration includes a determining unit 110, an obtaining unit 111, and a filtering unit 112. A module/unit as referred to herein is a series of computer program segments stored in a memory that can be executed by a processor and that can perform a fixed function. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
In this embodiment, the data filtering apparatus 11 based on database migration specifically includes:
in response to the system object data filtering instruction, the determination unit 110 determines the database type of the source database.
In at least one embodiment of the present invention, the source database may be any type of database, such as a relational database, for example: the source database may be an oracle database.
In at least one embodiment of the present invention, the system object data filtering instruction may be triggered automatically periodically, or may be triggered by a relevant worker when there is a migration requirement, which is not limited by the present invention.
In at least one embodiment of the present invention, the determining unit 110 determines the database type of the source database, including:
acquiring a preset parent table in the source database;
determining database version information supported by the source database according to the parent table;
and acquiring the database type from the database version information.
The database version information may include information such as a database type and a database version. For example: the database version information may be: database type oracle, version 11G.
The obtaining unit 111 obtains the system object data in the source database according to the database type, and loads the system object data to the configuration cache.
In at least one embodiment of the present invention, the obtaining unit 111 obtains the system object data in the source database according to the database type, including:
determining a child table associated with the parent table;
and acquiring the system object data from the sub-table according to the database type.
The child table is used for storing the system object data corresponding to each database type in the parent table.
The system object data is the non-service data carried by the source database.
The system object data may include, but is not limited to: system user, system tablespace, system role, system object list.
For example: the system user may be: a SYSTEM; the system tablespace may be: a SYSTEM; the system roles may be: DBA.
In this embodiment, one database type corresponds to multiple system users, multiple system tablespaces, and multiple system roles.
It can be understood that, since the system user, the system tablespace, and the system role are non-service data of the source database itself, and system data belonging to the source database itself has no service value, there is no need to perform database migration, otherwise, not only the efficiency of database migration is affected, but also data redundancy is caused.
In at least one embodiment of the present invention, before the determining the database type of the source database, the parent table and the child table are maintained by initializing an insert statement; or
And acquiring information input on a configuration interface, and maintaining the parent table and the child table according to the acquired information.
Wherein the insert statement is used to insert data into the parent table or the child table, such as: an insert statement in an SQL (Structured Query Language) script is initialized to insert database version information "database type oracle, version 11G" into the parent table.
The configuration interface may be an interface for interacting with a user, for example: and related operators can delete and add data on the configuration interface so as to maintain the parent table according to the requirements of users.
The parent table and the child table are maintained through the embodiment, so that the subsequent query can be used conveniently.
In at least one embodiment of the present invention, the configuration cache may be a temporary cache such as a memory of an associated server.
By loading the acquired system object data to the configuration cache, the data reading efficiency can be improved, and the database migration efficiency is further improved.
The obtaining unit 111 obtains the data to be migrated in the source database, and obtains the system object data in the configuration cache.
In this embodiment, the data to be migrated may include all migratable data in the source database.
The embodiment reads the system object data in the configuration cache, and can ensure the efficiency of data reading.
The filtering unit 112 filters the system object data in the data to be migrated to obtain target data suitable for migration.
In at least one embodiment of the present invention, the filtering unit 112 filters the system object data in the data to be migrated to obtain target data suitable for migration, which includes, but is not limited to, one or more of the following combinations:
(1) And when the system object data is a system user, querying the database user table of the source database, and eliminating the system user when querying the database user table of the source database to obtain a service user as the target data.
For example: when the source database is an oracle database, the database user table may be dba _ users.
In this example, the system user behaves as a "blacklisted user" and does not need to be migrated as it is non-traffic data.
Therefore, after the system user is excluded from the database user table, the remaining service users are service-related users and need to be migrated with the database for subsequent use.
(2) And when the system object data is a system table space, inquiring a database table space table of the source database, and eliminating the system table space when inquiring the database table space table of the source database to obtain a service table space as the target data.
For example: when the source database is an oracle database, the database table space table may be dba _ tables.
In this example, the system tablespace corresponds to a "blacklist database space," which, because it is non-business data, does not need to be migrated.
Therefore, after the system table space is excluded from the database table space table, the remaining service table space is a service-related table space and needs to be migrated with the database for subsequent use.
(3) And when the system object data is in a system role, inquiring the database role table of the source database, and eliminating the system role when inquiring the database role table of the source database to obtain a service role as the target data.
For example: when the source database is an oracle database, the database role table may be dba _ roles.
In this example, the system role is equivalent to a "blacklist database role," which, because it is non-business data, does not require migration.
Therefore, after the system role is excluded from the database roles, the remaining service roles are service-related roles and need to be migrated with the database for subsequent use.
Through the embodiment, the system object data can be automatically filtered from the data to be migrated, so that unnecessary data migration is avoided.
Referring further to fig. 2, when the system object data is a system object list, the filtering unit 112 filters the system object data in the data to be migrated to obtain target data suitable for migration, including:
the filtering unit 112 queries the database object table of the source database.
For example: when the source database is an oracle database, the database object table may be dba _ objects.
The filtering unit 112 obtains the database user to which each list of objects to be migrated belongs in the database object table.
The database user to which each object list to be migrated belongs may be the system user or the service user.
The filtering unit 112 excludes the system object list belonging to the system user from each object list to be migrated, and obtains a service object list as the target data.
For example: the object list whose owner is the system user (i.e. excluding the system object list belonging to the system user) is excluded from each object list to be migrated, and the remaining object lists are all the object lists to be migrated (i.e. the business object lists).
In this example, the system object list is equivalent to a "blacklisted object list" that does not require migration because it is non-business data.
Therefore, after the system object list is excluded from each object list to be migrated, the remaining service object list is an object list related to a service, and needs to be migrated with the database, so as to be used later.
In at least one embodiment of the present invention, after the target data is obtained, a target database is determined;
migrating the target data from the source database to the target database.
The target database may be any type of database such as a dreams database.
Through the embodiment, the system data carried by the source database can be automatically screened out in the data migration process, other data such as service data and the like can be migrated to the target database, manual screening and filtering are not needed, the database migration efficiency is improved, errors caused by manual operation are effectively avoided, and the data migration accuracy is improved.
Moreover, due to the fact that automatic screening is carried out, relevant operators do not need to have relevant knowledge of the database, and meanwhile do not need to know which business data are contained in the database, operability of database migration is improved, and the threshold is lowered.
According to the technical scheme, the database type of the source database is determined in response to the system object data filtering instruction; acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache; acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache; and filtering the system object data in the data to be migrated to obtain target data suitable for migration. The invention can improve the efficiency of database migration and reduce the error rate.
Fig. 4 is a schematic structural diagram of a computer device for implementing the data filtering method based on database migration according to the preferred embodiment of the present invention.
The computer device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as a data filter based on database migration, stored in the memory 12 and executable on the processor 13.
It will be appreciated by those skilled in the art that the schematic diagram is merely an example of the computer device 1, and does not constitute a limitation to the computer device 1, the computer device 1 may be in a bus structure or a star structure, the computer device 1 may include more or less hardware or software than those shown, or different component arrangements, for example, the computer device 1 may further include an input and output device, a network access device, and the like.
It should be noted that the computer device 1 is only an example, and other electronic products that are now or may come into existence in the future, such as may be adapted to the present invention, should also be included within the scope of the present invention, and are hereby incorporated by reference.
The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the computer device 1, e.g. a removable hard disk of the computer device 1. The memory 12 may also be an external storage device of the computer device 1 in other embodiments, such as a plug-in removable hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the computer device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the computer device 1. The memory 12 may be used not only to store application software installed in the computer apparatus 1 and various types of data, such as codes of a data filter program based on database migration, etc., but also to temporarily store data that has been output or is to be output.
The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the computer device 1, connects various components of the whole computer device 1 by using various interfaces and lines, executes various functions of the computer device 1 and processes data by running or executing programs or modules (for example, executing a data filtering program based on database migration, etc.) stored in the memory 12, and calling data stored in the memory 12.
The processor 13 executes the operating system of the computer device 1 and various installed application programs. The processor 13 executes the application program to implement the steps of the above-mentioned embodiments of the data filtering method based on database migration, such as the steps shown in fig. 1.
Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer readable instruction segments capable of performing certain functions, which are used to describe the execution of the computer program in the computer device 1. For example, the computer program may be divided into a determination unit 110, an acquisition unit 111, a filtering unit 112.
The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a processor (processor) to execute parts of the data filtering method based on database migration according to various embodiments of the present invention.
The modules/units integrated by the computer device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments described above may be implemented.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random-access Memory, or the like.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one line is shown in FIG. 4, but it is not intended that there be only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.
Although not shown, the computer device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so that functions of charge management, discharge management, power consumption management and the like are realized through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The computer device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the computer device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the computer device 1 and other computer devices.
Optionally, the computer device 1 may further comprise a user interface, which may be a Display (Display), an input unit, such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the computer device 1 and for displaying a visualized user interface.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
Fig. 4 shows only a computer device 1 with components 12-13, and it will be understood by those skilled in the art that the structure shown in fig. 4 does not constitute a limitation of the computer device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
With reference to fig. 1, the memory 12 of the computer device 1 stores a plurality of instructions to implement a database migration-based data filtering method, and the processor 13 can execute the plurality of instructions to implement:
responding to a system object data filtering instruction, and determining a database type of a source database;
acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache;
acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache;
and filtering the system object data in the data to be migrated to obtain target data suitable for migration.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1-2, which is not repeated herein.
It should be noted that all data involved in the present application are legally acquired.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The invention is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the present invention may also be implemented by one unit or means through software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A method for filtering data based on database migration, the method comprising:
responding to the system object data filtering instruction, and determining the database type of the source database;
acquiring system object data in the source database according to the database type, and loading the system object data to a configuration cache;
acquiring data to be migrated in the source database and acquiring the system object data in the configuration cache;
and filtering the system object data in the data to be migrated to obtain target data suitable for migration.
2. The method for database migration-based data filtering according to claim 1, wherein said determining the database type of the source database comprises:
acquiring a preset parent table in the source database;
determining database version information supported by the source database according to the parent table;
and acquiring the database type from the database version information.
3. The database migration-based data filtering method according to claim 2, wherein the obtaining system object data in the source database according to the database type includes:
determining a child table associated with the parent table;
and acquiring the system object data from the sub-table according to the database type.
4. The method for database migration-based data filtering according to claim 3, wherein prior to said determining the database type of the source database, said method further comprises:
maintaining the parent table and the child table by initializing insert statements; or
And acquiring information input on a configuration interface, and maintaining the parent table and the child table according to the acquired information.
5. The database migration-based data filtering method according to claim 1, wherein the filtering the system object data in the data to be migrated to obtain target data suitable for migration comprises:
when the system object data is a system user, inquiring the database user table of the source database, and eliminating the system user when inquiring the database user table of the source database to obtain a service user as the target data; and/or
When the system object data is a system table space, inquiring a database table space table of the source database, and eliminating the system table space when inquiring the database table space table of the source database to obtain a service table space as the target data; and/or
And when the system object data is in a system role, inquiring the database role table of the source database, and eliminating the system role when inquiring the database role table of the source database to obtain a service role as the target data.
6. The database migration-based data filtering method according to claim 5, wherein the filtering the system object data in the data to be migrated to obtain target data suitable for migration comprises:
when the system object data is a system object list, querying a database object table of the source database;
acquiring a database user to which each object list to be migrated belongs in the database object table;
and excluding the system object list belonging to the system user from each object list to be migrated to obtain a service object list as the target data.
7. The database migration-based data filtering method according to claim 1, wherein after said obtaining the target data, the method further comprises:
determining a target database;
migrating the target data from the source database to the target database.
8. An apparatus for filtering data based on database migration, the apparatus comprising:
the determining unit is used for responding to the system object data filtering instruction and determining the database type of the source database;
the acquisition unit is used for acquiring the system object data in the source database according to the database type and loading the system object data to a configuration cache;
the obtaining unit is further configured to obtain data to be migrated in the source database, and obtain the system object data in the configuration cache;
and the filtering unit is used for filtering the system object data in the data to be migrated so as to obtain target data suitable for migration.
9. A computer device, characterized in that the computer device comprises:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the database migration-based data filtering method of any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executable by a processor in a computer device to implement the database migration-based data filtering method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210959015.5A CN115292289A (en) | 2022-08-09 | 2022-08-09 | Data filtering method, device, equipment and medium based on database migration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210959015.5A CN115292289A (en) | 2022-08-09 | 2022-08-09 | Data filtering method, device, equipment and medium based on database migration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115292289A true CN115292289A (en) | 2022-11-04 |
Family
ID=83828876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210959015.5A Pending CN115292289A (en) | 2022-08-09 | 2022-08-09 | Data filtering method, device, equipment and medium based on database migration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115292289A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170046143A1 (en) * | 2015-08-12 | 2017-02-16 | Oracle International Corporation | Peoplesoft cloud manager |
| CN111984621A (en) * | 2020-07-17 | 2020-11-24 | 新华三大数据技术有限公司 | Heterogeneous database migration method and device and storage medium |
| CN112434004A (en) * | 2020-10-28 | 2021-03-02 | 金蝶云科技有限公司 | Data migration method and device for heterogeneous system, computer equipment and storage medium |
| CN113297180A (en) * | 2021-06-01 | 2021-08-24 | 平安证券股份有限公司 | Data migration method and device, electronic equipment and storage medium |
-
2022
- 2022-08-09 CN CN202210959015.5A patent/CN115292289A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170046143A1 (en) * | 2015-08-12 | 2017-02-16 | Oracle International Corporation | Peoplesoft cloud manager |
| CN111984621A (en) * | 2020-07-17 | 2020-11-24 | 新华三大数据技术有限公司 | Heterogeneous database migration method and device and storage medium |
| CN112434004A (en) * | 2020-10-28 | 2021-03-02 | 金蝶云科技有限公司 | Data migration method and device for heterogeneous system, computer equipment and storage medium |
| CN113297180A (en) * | 2021-06-01 | 2021-08-24 | 平安证券股份有限公司 | Data migration method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112115143A (en) | Automatic data updating and synchronizing method and device, electronic equipment and storage medium | |
| CN112364107A (en) | System analysis visualization method and device, electronic equipment and computer readable storage medium | |
| CN111538573A (en) | Asynchronous task processing method and device and computer readable storage medium | |
| CN112115152A (en) | Data increment updating and querying method and device, electronic equipment and storage medium | |
| CN115408399A (en) | Blood relationship analysis method, device, equipment and storage medium based on SQL script | |
| CN114610747A (en) | Data query method, device, equipment and storage medium | |
| CN113806434A (en) | Big data processing method, device, equipment and medium | |
| CN112256783A (en) | Data export method and device, electronic equipment and storage medium | |
| CN115617403A (en) | Clearing task execution method, device, equipment and medium based on task segmentation | |
| CN113590632B (en) | Database index creation method, device, equipment and medium | |
| CN111352955A (en) | Index data query method and device, electronic equipment and storage medium | |
| CN114691050A (en) | Cloud native storage method, device, equipment and medium based on kubernets | |
| CN114880368A (en) | Data query method and device, electronic equipment and readable storage medium | |
| CN113297180A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN113434542A (en) | Data relation identification method and device, electronic equipment and storage medium | |
| CN112948380A (en) | Data storage method and device based on big data, electronic equipment and storage medium | |
| CN114816371B (en) | Message processing method, device, equipment and medium | |
| CN115292289A (en) | Data filtering method, device, equipment and medium based on database migration | |
| CN114116673A (en) | Data migration method based on artificial intelligence and related equipment | |
| CN114547011A (en) | Data extraction method and device, electronic equipment and storage medium | |
| CN113935663A (en) | Equipment combination analysis method, device, equipment and medium for panel product | |
| CN113609428A (en) | Method, device, equipment and medium for online checking prototype design file based on Git | |
| CN113918305A (en) | Node scheduling method and device, electronic equipment and readable storage medium | |
| CN114860314B (en) | Deployment upgrading method, device, equipment and medium based on database compatibility | |
| CN115065642B (en) | Code table request method, device, equipment and medium under bandwidth limitation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |