CN114610782A - Data source switching method and device, electronic equipment and storage medium - Google Patents

Data source switching method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN114610782A
CN114610782A CN202210287031.4A CN202210287031A CN114610782A CN 114610782 A CN114610782 A CN 114610782A CN 202210287031 A CN202210287031 A CN 202210287031A CN 114610782 A CN114610782 A CN 114610782A
Authority
CN
China
Prior art keywords
data
metadata
switched
data source
similarity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210287031.4A
Other languages
Chinese (zh)
Inventor
孙佳兴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OneConnect Financial Technology Co Ltd Shanghai
Original Assignee
OneConnect Financial Technology Co Ltd Shanghai
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by OneConnect Financial Technology Co Ltd Shanghai filed Critical OneConnect Financial Technology Co Ltd Shanghai
Priority to CN202210287031.4A priority Critical patent/CN114610782A/en
Publication of CN114610782A publication Critical patent/CN114610782A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2457—Query processing with adaptation to user needs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models

Abstract

The embodiment of the application discloses a data source switching method and device, electronic equipment and a storage medium. The method comprises the following steps: based on the current data source, constructing a data model layer corresponding to the current data source, wherein the data model layer comprises instance data and metadata corresponding to the instance data; acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched; and determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity. The problem of inaccurate association caused by manual association is avoided, and the problem that only pre-established standby data sources can be switched can also be avoided.

Description

Data source switching method and device, electronic equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of data source switching, in particular to a data source switching method and device, electronic equipment and a storage medium.
Background
In a data intensive application scenario, a data source is the basis of platform operation, but due to instability of a data provider that provides the data source for a platform and uncertainty of a contract, the data source often changes, for example, from the data source of provider a to the data source of provider B, and at this time, for the platform, switching of the data source is required, that is, a new data source is mapped into the platform, so that various applications in the platform operation process are communicated with the new data source.
Generally, in the prior art, a data model layer is established, and then a new data source and the data model layer are manually associated, or a standby data source is established. However, in the two manners, the workload of manual association is large, and there may be an influence on subjective understanding, which causes inaccurate association, and establishing the backup data source only can switch the data source to the backup data source, but the purpose of switching to the new data source cannot be achieved when there is a new data source.
Disclosure of Invention
The embodiment of the application provides a data source switching method and device, electronic equipment and a storage medium, so as to improve data security of data source switching.
In a first aspect, an embodiment of the present application provides a data source switching method, which is applied to a server, and the method includes:
constructing a data model layer corresponding to a current data source based on the current data source, wherein the data model layer comprises example data and metadata corresponding to the example data;
acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched;
and determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity.
In a second aspect, an embodiment of the present application further provides a data source switching apparatus, where the data source switching apparatus includes:
the data model building module is used for building a data model layer corresponding to a current data source based on the current data source, and the data model layer comprises example data and metadata corresponding to the example data;
the metadata acquisition module is used for acquiring to-be-switched data of a to-be-switched data source and metadata corresponding to the to-be-switched data;
and the switching module is used for determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair and switching the current data source into the data source to be switched according to the similarity.
In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the data source switching method provided by any embodiment of the application.
In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the data source switching method according to any embodiment of the present application.
According to the technical scheme of the embodiment of the application, the metadata capable of describing the example data and the data to be switched are obtained, the similarity between the metadata of the example data and the metadata of the data to be switched is determined, and the current data source is switched to the data source to be switched according to the similarity. The method can avoid the problem of inaccurate association caused by manual association and also can avoid the problem that only the pre-established standby data source can be switched.
Drawings
Fig. 1 is a schematic flowchart of a data source switching method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of building a data model layer according to a first embodiment of the present application;
fig. 3 is a schematic flow chart illustrating data source switching according to similarity according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data source switching device according to a second embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to a third embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a schematic flow chart of a data source switching method according to an embodiment of the present application, which is applicable to a data source switching scenario and applied to a server. The method can be executed by a data source switching device, which can be implemented in a hardware and/or software manner and can be generally integrated into an electronic device such as a computer with data operation capability, and specifically includes the following steps:
step 101, building a data model layer corresponding to the current data source based on the current data source, wherein the data model layer comprises instance data and metadata corresponding to the instance data.
In this step, the current data source refers to a data source to be disconnected, and the reason for the disconnection may be that the data supply system of the data source provider is unstable or the time agreed with the data source provider expires. It should be noted that, because there are many reasons for the outage, for each reason, a monitor may be correspondingly set to monitor whether the reason occurs.
For example, for the reason that the data supply system of the data source provider is unstable, the time required for data retrieval may be monitored, and if the time duration is greater than a certain time threshold, it may be determined that the data source is about to be disconnected, and the method steps of the present application may be executed.
For another example, for the reason that the time agreed with the data source provider expires, a countdown timer may be set as a monitor, and if the remaining time of the countdown is less than a certain time threshold, it may be determined that the data source is about to be powered off, and the method steps of the present application may be started.
In addition, in this step, for the construction of the data model layer, refer to fig. 2, and fig. 2 is a schematic flow chart of constructing the data model layer according to the first embodiment of the present application.
As shown in fig. 2, the process of building a data model layer provided in this embodiment may include:
step 201, obtaining a target data field where a current data source is located, and determining a target standard metadata table corresponding to the target data field according to a mapping relation between the pre-established data field and the standard metadata table.
It should be noted that the data field refers to a field to which data is applied, and in this step, the field in which the application system is located may be directly obtained. The application system refers to a system which calls a data source to meet user requirements, such as search requirements and the like.
In addition, the field of the application system is predetermined, such as the financial field, the medical field, the scientific and technological field, and the like, and generally, the information of the field can be stored in a certain storage location of the application system in advance and can be directly called.
Each data field is provided with a standard metadata table for establishing a data model layer of the data field more conveniently. In this step, a mapping relationship between the data field and the standard metadata table may be stored in the application system in advance, and may be specifically as shown in table 1 below.
TABLE 1
Data field Standard metadata tables Obtaining a path
The field of finance TABLE a Route 1
Medical field Table b Route 2
Field of intelligence Table c Route 3
...... ...... ......
In a specific example, if the obtained target data field is an intelligent field, it can be known from the mapping relationship in table 1 that table c is a target standard metadata table corresponding to the target data field. In order to reduce the occupation of the storage space in the application system, a corresponding acquisition path can be further added in the mapping relation, and the acquisition path can point to the cloud address storing the corresponding standard metadata table.
Still taking the foregoing example as an example, after determining that the target standard metadata table is table c, table c may be obtained through path 3 corresponding to table c.
And 202, calculating the similarity between the metadata in the target standard metadata table and each instance data in the current data source.
In this step, the target standard metadata table includes many standard metadata, and the data source includes many instance data. Therefore, this step can be calculated pair by pair. In order to make the calculation process more standard, the calculation may be performed based on the metadata in the target standard metadata table, or based on the instance data in the current data source.
In a specific example, the target standard metadata table includes five metadata of a1, B1, C1, D1 and E1, and the current data source includes five instance data of a2, B2, C2, D2 and E2. If the metadata in the target standard metadata table is taken as a reference, first metadata A1 is obtained from the target standard metadata table, and then the metadata A is sequentially grouped with the example data in the current data source to form: a1, A2; a1, B2; a1, C2; a1, D2; a1, E2. And then calculating corresponding similarity pair by pair, then acquiring second metadata B1 from the target standard metadata table, and sequentially grouping the second metadata B1 with example data in the current data source to calculate the similarity pair by pair until all metadata in the target standard metadata table are acquired.
If the example data in the current data source is taken as a reference, first example data A2 is obtained from the current data source, and then the first example data A2 is sequentially queued with the metadata in the target standard metadata table to form: a2, A1; a2, B1; a2, C1; a2, D1; a2, E1. And then calculating the similarity of each pair by pair, after the 5 pairs are calculated, acquiring a second example data B2 from the current data source, and forming a queue with the metadata in the target standard metadata table in sequence to calculate the similarity pair by pair until all the example data in the current data source are acquired.
In calculating the similarity, since the metadata is often data containing description information and is different from the representation form of the example data, the example data is summarized to obtain a summarized text of the example data. It should be noted that, the generalized manner may be: the main data content is extracted from the instance data and then converted to text.
After the summarized text is obtained, the similarity between the summarized text and each metadata in the target standard metadata table can be determined, and the obtained similarity is determined as the similarity between the example data and each metadata in the target standard metadata table.
And step 203, mapping the metadata in the target standard metadata table and the example data of the current data source according to the similarity.
And step 204, generating a data model layer based on the mapping relation between the metadata in the target standard metadata table and each instance data.
In the above step, the metadata with the highest similarity may be mapped with the instance data, and still taking the above example as an example, the instance data in the current data source is taken as a reference, and the metadata is formed into a2 and a 1; a2, B1; a2, C1; a2, D1; a2, E1. The similarity of each pair is calculated, and then the pair of metadata and the instance data with the highest similarity are mapped. After the mapping is finished, all the mappings can be combined to form a data model layer.
102, acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched.
It should be noted that the data source to be switched is the data source to be switched, that is, the current data source is switched to the data source to be switched in this step. The data source to be switched can be provided by a data provider, and the data source to be switched provided by the data provider contains the data to be switched and metadata for describing the definition of the data to be switched.
The data source to be switched contains a plurality of data to be switched, and any one piece of data to be switched has corresponding metadata. Specific examples are shown in table 2 below.
TABLE 2
Data to be switched Metadata
Data to be switched a1 Metadata a1
Data to be switched b1 Metadata b1
Data to be switched c1 Metadatac1
...... ......
It should be noted that, in this step, the data provided by the data provider may not be stored in a table form by mapping, but may be mapped in a key pair manner, or may be mapped in another manner.
And 103, determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity.
In this step, when pair-by-pair determination is performed, the metadata of the data to be switched and the metadata of the instance data need to be paired, and when pairing is performed, the metadata of the data to be switched can be paired with the metadata of the instance data by taking the metadata of the instance data as a reference, and the metadata of the instance data can also be paired with the metadata of the data to be switched by taking the metadata of the instance data as a reference.
In one specific example, the metadata of the data to be switched may include: metadata a1, metadata b1 and metadata c1, and metadata a2, metadata b2 and metadata c2 are examples of metadata of the example data. If the metadata of the data to be switched is taken as a reference, the metadata a1 is taken out first, and the metadata is sequentially paired with the metadata of the example data to obtain the following three pairs: metadata a1, metadata a 2; metadata a1, metadata b 2; metadata a1, metadata c 2.
And taking out metadata b1, and sequentially pairing the metadata with metadata of the example data to obtain the following three pairs: metadata b1, metadata a 2; metadata b1, metadata b 2; metadata b1, metadata c 2. Metadata c1 is fetched and paired with metadata of the instance data in order to get the following three pairs: metadata c1, metadata a 2; metadata c1, metadata b 2; metadata c1, metadata c 2. Since the metadata of the data to be switched is already taken out, the pairing is completed, and the following 9 pairs are obtained:
metadata a1, metadata a 2; metadata a1, metadata b 2; metadata a1, metadata c 2; metadata b1, metadata a 2; metadata b1, metadata b 2; metadata b1, metadata c 2; metadata c1, metadata a 2; metadata c1, metadata b 2; metadata c1, metadata c 2.
Of course, if the metadata of the example data is taken as a reference, the metadata a2 is taken out first, and the metadata is sequentially paired with the metadata of the data to be switched, so as to obtain the following three pairs: metadata a2, metadata a 1; metadata a2, metadata b 1; metadata a2, metadata c 1.
And taking out the metadata b2, and sequentially pairing the metadata b2 with the metadata of the data to be switched to obtain the following three pairs: metadata b2, metadata a 1; metadata b2, metadata b 1; metadata b2, metadata c 1. And taking out metadata c2, and sequentially pairing the metadata with metadata of data to be switched to obtain the following three pairs: metadata c2, metadata a 1; metadata c2, metadata b 1; metadata c2, metadata c 1. Since the metadata of the data to be switched is already taken out, the pairing is completed, and the following 9 pairs are obtained:
metadata a2, metadata a 1; metadata a2, metadata b 1; metadata a2, metadata c 1; metadata b2, metadata a 1; metadata b2, metadata b 1; metadata b2, metadata c 1; metadata c2, metadata a 1; metadata c2, metadata b 1; metadata c2, metadata c 1.
No matter which way is used for pairing, the pairing result is the same, and two groups of metadata are paired one by one to ensure that all possible pairing ways are matched.
When the similarity is calculated specifically, for any metadata of data to be switched and metadata of example data, the metadata of the data to be switched and the metadata of the example data are converted into word vectors respectively; and calculating the similarity between the converted word vectors, and determining the calculation result as the similarity between the metadata of the data to be switched and the metadata of the instance data.
The process of converting the metadata into word vectors may specifically be that, for any metadata of data to be converted and metadata of instance data, the metadata is summarized to obtain a summarized text of the metadata; and converting the summarized text of the metadata into a word vector by contrasting a preset word vector dictionary.
It should be noted that, since the metadata is descriptive information and the content of the metadata may be more, the content of the metadata may be summarized first. The generalized manner may be keyword extraction, and the specific algorithm for keyword extraction may be an existing related algorithm, which is not described herein again.
In addition, for the convenience of subsequent similarity calculation, the summarized text can be quantized in a way of converting word vectors, namely converting the summarized text of the metadata into the word vectors. It should be noted that, when converting a word vector, a preset word vector dictionary may be referred to, vector values into which each word can be converted are agreed in the word vector dictionary, and the words in the summarized text may be converted one by referring to the word vector dictionary.
In a specific example, the summary text may be "user, address", a word vector corresponding to "use" may be found from a word vector dictionary, then a word vector of "user", then a word vector of "live", and finally a word vector of "address", and then the four word vectors are spliced into a word vector of the summary text. It should be noted that the way of splicing may be to splice them into a matrix, where each row in the matrix represents a word.
Further, when calculating the similarity of the word vectors, the distance between the converted word vectors may be calculated according to a preset vector distance formula, and the distance may be determined as the similarity between the converted word vectors.
Specifically, the preset vector distance formula may be as follows:
Figure BDA0003558666290000101
wherein d (i, j) represents a word vector YiAnd the word vector YjThe distance between them.
After the similarity is calculated, the data source may be switched, specifically referring to fig. 3, where fig. 3 is a schematic flow diagram of switching the data source according to the similarity according to an embodiment of the present application.
As shown in fig. 3, the process of switching data sources according to the similarity may include:
step 301, for any data to be switched, determining target instance data having the highest similarity with a data source to be switched.
In this step, the metadata pair with the highest similarity is found from the metadata pairs containing the data to be switched, the metadata of the instance data contained in the metadata pair corresponding to the highest similarity is the metadata that is to be found in this step, and the corresponding instance data is the target instance data in this step.
In one specific example, the metadata pair containing metadata a1 of the data to be switched is: metadata a1, metadata a 2; metadata a1, metadata b 2; if the calculated similarities of the metadata a1 and the metadata c2 are 0.6, 0.95, and 0.8 in this order, it can be stated that the metadata pair: the metadata a1 and the metadata b2 have the highest similarity, that is, the target instance data with the highest similarity to the data a1 to be switched is the instance data corresponding to the metadata b 2.
Step 302, obtaining a mapping relation between the target instance data and the data application layer, where the mapping relation includes a storage address of the target instance data.
In this step, when the data application layer calls data, it usually calls corresponding instance data through a storage address, and therefore when the data application layer calls data, it needs to first obtain the storage address of the instance data that it wants to call, and the obtaining way is the mapping relationship between the instance data and the data application layer, and in the mapping relationship, the mapping relationship between the function of the data application layer and the storage address of the instance data, such as a search function, maps a plurality of storage addresses of the instance data that can be obtained through search.
Therefore, in this step, the storage address of the target instance data may be obtained first, and then the mapping including the storage address, that is, the mapping between the target instance data and the data application layer, obtained in this step, is found from the mapping between the instance data and the data application layer.
And 303, changing the storage address in the mapping relation into the storage address of the data to be switched so as to switch the data source corresponding to the data application layer into the data source to be switched.
In this step, the storage address in the mapping relationship is directly changed into the storage address of the data to be switched, so that when the data application layer calls the data, the data to be switched corresponding to the changed storage address is called, and the switching of the data source is completed.
In the embodiment, the similarity between the metadata of the example data and the metadata of the data to be switched is determined by acquiring the metadata with the capability of describing the example data and the data to be switched, and the current data source is switched to the data source to be switched according to the similarity. The method can avoid the problem of inaccurate association caused by manual association and also can avoid the problem that only the pre-established standby data source can be switched.
Example two
Fig. 4 is a schematic structural diagram of a data source switching device according to a second embodiment of the present application. The data source switching device provided by the embodiment of the application can execute the data source switching method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of the execution method. The apparatus may be implemented in a software and/or hardware manner, and as shown in fig. 4, the data source switching apparatus specifically includes: a data model building module 401, a metadata obtaining module 402, and a switching module 403.
The data model building module is used for building a data model layer corresponding to the current data source based on the current data source, and the data model layer comprises instance data and metadata corresponding to the instance data;
the metadata acquisition module is used for acquiring the data to be switched of the data source to be switched and metadata corresponding to the data to be switched;
and the switching module is used for determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair and switching the current data source into the data source to be switched according to the similarity.
In the embodiment, the similarity between the metadata of the example data and the metadata of the data to be switched is determined by acquiring the metadata capable of describing the example data and the data to be switched, and the current data source is switched to the data source to be switched according to the similarity. The method can avoid the problem of inaccurate association caused by manual association and also can avoid the problem that only the pre-established standby data source can be switched.
Further, the data model building module comprises:
the target standard metadata table determining unit is used for acquiring a target data field where a current data source is located and determining a target standard metadata table corresponding to the target data field according to a mapping relation between the pre-established data field and the standard metadata table;
the first similarity calculation unit is used for calculating the similarity between the metadata in the target standard metadata table and each instance of data in the current data source;
the mapping unit is used for mapping the metadata in the target standard metadata table and the example data of the current data source according to the similarity;
and the data model layer generating unit is used for generating a data model layer based on the mapping relation between the metadata in the target standard metadata table and each instance data.
Further, the similarity calculation unit includes:
the summarizing text determining subunit is used for summarizing the example data to obtain a summarizing text of the example data;
and the similarity determining subunit is used for determining the similarity between the summarized text and each metadata in the target standard metadata table, and determining the obtained similarity as the similarity between the example data and each metadata in the target standard metadata table.
Further, the switching module includes:
the word vector conversion unit is used for respectively converting the metadata of the data to be switched and the metadata of the example data into word vectors for any metadata of the data to be switched and the metadata of the example data;
and the second similarity calculation unit is used for calculating the similarity between the converted word vectors and determining the calculation result as the similarity between the metadata of the data to be switched and the metadata of the instance data.
Further, the second similarity degree calculation unit includes:
and the distance calculating subunit is used for calculating the distance between the converted word vectors according to a preset vector distance formula and determining the distance as the similarity between the converted word vectors.
Further, the word vector conversion unit includes:
the summarizing subunit is used for summarizing any metadata in the metadata of the data to be switched and the metadata of the example data to obtain a summarizing text of the metadata;
and the conversion subunit is used for converting the summarized text of the metadata into word vectors by contrasting a preset word vector dictionary.
Further, the switching module includes:
the target example data determining unit is used for determining target example data with the highest similarity to the data source to be switched for any data to be switched;
the storage address determining unit is used for acquiring a mapping relation between target instance data and a data application layer, wherein the mapping relation comprises a storage address of the target instance data;
and the address changing unit is used for changing the storage address in the mapping relation into the storage address of the data to be switched so as to switch the data source corresponding to the data application layer into the data source to be switched.
EXAMPLE III
Fig. 5 is a schematic structural diagram of an electronic device according to a third embodiment of the present application, as shown in fig. 5, the electronic device includes a processor 510, a memory 520, an input device 530, and an output device 540; the number of the processors 510 in the electronic device may be one or more, and one processor 510 is taken as an example in fig. 5; the processor 510, the memory 520, the input device 530 and the output device 540 in the electronic apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 5.
The memory 520 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the data source switching method in the embodiment of the present invention (for example, the receiving module 301, the signature verifying module 302, the decrypting module 303, and the data feedback module 305 in the data source switching apparatus). The processor 510 executes various functional applications and data processing of the electronic device by executing software programs, instructions and modules stored in the memory 520, that is, the data source switching method described above is implemented:
based on the current data source, constructing a data model layer corresponding to the current data source, wherein the data model layer comprises instance data and metadata corresponding to the instance data;
acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched;
and determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity.
Further, constructing a data model layer corresponding to the current data source based on the current data source includes:
acquiring a target data field where a current data source is located, and determining a target standard metadata table corresponding to the target data field according to a mapping relation between the pre-established data field and the standard metadata table;
calculating the similarity between the metadata in the target standard metadata table and each instance data in the current data source;
mapping the metadata in the target standard metadata table and the example data of the current data source according to the similarity;
and generating a data model layer based on the mapping relation between the metadata in the target standard metadata table and each instance data.
Further, calculating the similarity between the metadata in the target standard metadata table and each instance of data in the current data source includes:
summarizing the example data to obtain a summarized text of the example data for any example data;
and determining the similarity between the summarized text and each metadata in the target standard metadata table, and determining the obtained similarity as the similarity between the example data and each metadata in the target standard metadata table.
Further, determining similarity between the metadata of the data to be switched and the metadata of the instance data pair by pair includes:
for any metadata of data to be switched and metadata of instance data, respectively converting the metadata of the data to be switched and the metadata of the instance data into word vectors;
and calculating the similarity between the converted word vectors, and determining the calculation result as the similarity between the metadata of the data to be switched and the metadata of the instance data.
Further, calculating the similarity between the converted word vectors includes:
and calculating the distance between the converted word vectors according to a preset vector distance formula, and determining the distance as the similarity between the converted word vectors.
Further, converting the metadata of the data to be converted and the metadata of the instance data into word vectors respectively, includes:
summarizing any metadata in metadata of the data to be switched and metadata of the instance data to obtain a summarized text of the metadata;
and converting the summarized text of the metadata into a word vector by contrasting a preset word vector dictionary.
Further, switching the current data source to the data source to be switched according to the similarity, including:
for any data to be switched, determining target instance data with the highest similarity to a data source to be switched;
acquiring a mapping relation between target instance data and a data application layer, wherein the mapping relation comprises a storage address of the target instance data;
and changing the storage address in the mapping relation into the storage address of the data to be switched so as to switch the data source corresponding to the data application layer into the data source to be switched.
The memory 520 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 520 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 520 may further include memory located remotely from processor 510, which may be connected to an electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Example four
A fourth embodiment of the present application further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a data source switching method, where the method includes:
based on the current data source, constructing a data model layer corresponding to the current data source, wherein the data model layer comprises instance data and metadata corresponding to the instance data;
acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched;
and determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity.
Further, constructing a data model layer corresponding to the current data source based on the current data source includes:
acquiring a target data field where a current data source is located, and determining a target standard metadata table corresponding to the target data field according to a mapping relation between the pre-established data field and the standard metadata table;
calculating the similarity between the metadata in the target standard metadata table and each instance data in the current data source;
mapping the metadata in the target standard metadata table and the example data of the current data source according to the similarity;
and generating a data model layer based on the mapping relation between the metadata in the target standard metadata table and each instance data.
Further, calculating the similarity between the metadata in the target standard metadata table and each instance of data in the current data source includes:
summarizing the example data to obtain a summarized text of the example data for any example data;
and determining the similarity between the summarized text and each metadata in the target standard metadata table, and determining the obtained similarity as the similarity between the example data and each metadata in the target standard metadata table.
Further, determining similarity between the metadata of the data to be switched and the metadata of the instance data pair by pair includes:
for any metadata of data to be switched and metadata of instance data, respectively converting the metadata of the data to be switched and the metadata of the instance data into word vectors;
and calculating the similarity between the converted word vectors, and determining the calculation result as the similarity between the metadata of the data to be switched and the metadata of the instance data.
Further, calculating the similarity between the converted word vectors includes:
and calculating the distance between the converted word vectors according to a preset vector distance formula, and determining the distance as the similarity between the converted word vectors.
Further, converting the metadata of the data to be converted and the metadata of the instance data into word vectors respectively, includes:
summarizing any metadata in metadata of the data to be switched and metadata of the instance data to obtain a summarized text of the metadata;
and converting the summarized text of the metadata into a word vector by contrasting a preset word vector dictionary.
Further, switching the current data source to the data source to be switched according to the similarity, including:
for any data to be switched, determining target instance data with the highest similarity to a data source to be switched;
acquiring a mapping relation between target instance data and a data application layer, wherein the mapping relation comprises a storage address of the target instance data;
and changing the storage address in the mapping relation into the storage address of the data to be switched so as to switch the data source corresponding to the data application layer into the data source to be switched.
Of course, the storage medium provided in the embodiments of the present application and containing computer-executable instructions is not limited to the above method operations, and may also perform related operations in the data source switching method provided in any embodiment of the present application.
From the above description of the embodiments, it is obvious for those skilled in the art that the present application can be implemented by software and necessary general hardware, and certainly can be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods of the embodiments of the present application.
It should be noted that, in the embodiment of the above search apparatus, each included unit and module are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the application.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments illustrated herein, and that various obvious changes, rearrangements and substitutions may be made therein by those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims (10)

1. A data source switching method is applied to a server side, and is characterized in that the method comprises the following steps:
constructing a data model layer corresponding to a current data source based on the current data source, wherein the data model layer comprises example data and metadata corresponding to the example data;
acquiring data to be switched of a data source to be switched and metadata corresponding to the data to be switched;
and determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair, and switching the current data source into the data source to be switched according to the similarity.
2. The method of claim 1, wherein constructing the data model layer corresponding to the current data source based on the current data source comprises:
acquiring a target data field where a current data source is located, and determining a target standard metadata table corresponding to the target data field according to a mapping relation between the pre-established data field and the standard metadata table;
calculating the similarity between the metadata in the target standard metadata table and each instance data in the current data source;
mapping the metadata in the target standard metadata table and the instance data of the current data source according to the similarity;
and generating a data model layer based on the mapping relation between the metadata in the target standard metadata table and each instance data.
3. The method according to claim 2, wherein the calculating the similarity between the metadata in the target standard metadata table and each instance of data in the current data source comprises:
summarizing the example data to obtain a summarized text of the example data;
and determining the similarity between the summarized text and each metadata in the target standard metadata table, and determining the obtained similarity as the similarity between the example data and each metadata in the target standard metadata table.
4. The method of claim 1, wherein determining the similarity between the metadata of the data to be switched and the metadata of the instance data pair by pair comprises:
for any metadata of data to be switched and metadata of the instance data, respectively converting the metadata of the data to be switched and the metadata of the instance data into word vectors;
and calculating the similarity between the word vectors obtained by conversion, and determining the calculation result as the similarity between the metadata of the data to be switched and the metadata of the instance data.
5. The method of claim 4, wherein the calculating the similarity between the transformed word vectors comprises:
and calculating the distance between the word vectors obtained by conversion according to a preset vector distance formula, and determining the distance as the similarity between the word vectors obtained by conversion.
6. The method of claim 4, wherein converting metadata of the data to be switched and metadata of the instance data into word vectors, respectively, comprises:
summarizing any metadata in metadata of data to be switched and metadata of the example data to obtain a summarized text of the metadata;
and converting the summarized text of the metadata into a word vector by contrasting a preset word vector dictionary.
7. The method according to claim 1, wherein the switching the current data source to the data source to be switched according to the similarity comprises:
for any data to be switched, determining target instance data with the highest similarity to the data source to be switched;
acquiring a mapping relation between the target instance data and a data application layer, wherein the mapping relation comprises a storage address of the target instance data;
and changing the storage address in the mapping relation into the storage address of the data to be switched so as to switch the data source corresponding to the data application layer into the data source to be switched.
8. A data source switching apparatus, the apparatus comprising:
the data model building module is used for building a data model layer corresponding to a current data source based on the current data source, and the data model layer comprises instance data and metadata corresponding to the instance data;
the metadata acquisition module is used for acquiring to-be-switched data of a to-be-switched data source and metadata corresponding to the to-be-switched data;
and the switching module is used for determining the similarity between the metadata of the data to be switched and the metadata of the example data pair by pair and switching the current data source into the data source to be switched according to the similarity.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the data source switching method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a data source switching method according to any one of claims 1 to 7.
CN202210287031.4A 2022-03-22 2022-03-22 Data source switching method and device, electronic equipment and storage medium Pending CN114610782A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210287031.4A CN114610782A (en) 2022-03-22 2022-03-22 Data source switching method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210287031.4A CN114610782A (en) 2022-03-22 2022-03-22 Data source switching method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114610782A true CN114610782A (en) 2022-06-10

Family

ID=81864640

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210287031.4A Pending CN114610782A (en) 2022-03-22 2022-03-22 Data source switching method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114610782A (en)

Similar Documents

Publication Publication Date Title
CN112866391A (en) Message pushing method and device, electronic equipment and storage medium
CN113886434A (en) Database cluster-based query and storage method, device and equipment
CN114202027A (en) Execution configuration information generation method, model training method and device
CN112559631A (en) Data processing method and device of distributed graph database and electronic equipment
CN114090735A (en) Text matching method, device, equipment and storage medium
CN103530345A (en) Short text characteristic extension and fitting characteristic library building method and device
CN114417856B (en) Text sparse coding method and device and electronic equipment
CN114610782A (en) Data source switching method and device, electronic equipment and storage medium
CN113239054B (en) Information generation method and related device
CN114969444A (en) Data processing method and device, electronic equipment and storage medium
CN114357180A (en) Knowledge graph updating method and electronic equipment
CN113691403A (en) Topological node configuration method, related device and computer program product
CN113726885A (en) Method and device for adjusting flow quota
CN113961600A (en) Data query method and device, computer equipment and storage medium
CN116561106B (en) Configuration item data management method and system
US20230132173A1 (en) Data reading method, device and storage medium
CN113434508B (en) Method and apparatus for storing information
CN112783507B (en) Data stream guiding playback method and device, electronic equipment and readable storage medium
CN113946702A (en) Image feature storage method and device, storage medium and electronic equipment
CN117520432A (en) Template-based data transmission method, device, equipment and storage medium
CN116028529A (en) Database data processing method, device, equipment and storage medium
CN114722041A (en) Data duplicate checking method, device and medium
CN116204559A (en) Data storage and data query method and device, electronic equipment and medium
CN116186455A (en) Catalog generation method, catalog operation method, device and equipment
CN113268987A (en) Entity name identification method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination