CN111125472B - Result backtracking method and device, electronic equipment and storage medium - Google Patents
Result backtracking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111125472B CN111125472B CN201911387737.2A CN201911387737A CN111125472B CN 111125472 B CN111125472 B CN 111125472B CN 201911387737 A CN201911387737 A CN 201911387737A CN 111125472 B CN111125472 B CN 111125472B
- Authority
- CN
- China
- Prior art keywords
- data
- metadata
- backtracking
- data source
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a result backtracking method, a device, electronic equipment and a storage medium, comprising the following steps: receiving a first backtracking request sent by electronic equipment; responding to a first backtracking request, and transmitting all prestored model data to the electronic equipment for display; receiving a second backtracking request for backtracking the data to be backtracked in all model data, wherein the second backtracking request is sent by the electronic equipment and used for characterizing the data to be backtracked; the second backtracking request comprises an identifier of data to be backtracked; responding to a second backtracking request, and searching metadata corresponding to the identification in a first corresponding relation based on the prestored identification and metadata of a data source; inputting the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and sending the corresponding data sources to the electronic equipment so as to realize result backtracking when the data sources are more and the calculation is complex.
Description
Technical Field
The present invention relates to the field of information processing technologies, and in particular, to a result backtracking method, device, electronic apparatus, and storage medium.
Background
In a data modeling system, model data is obtained by processing one or more data sources using certain algorithms or operations, however, the result backtracking is to backtrack the data source associated with the model data based on the obtained model data after the model data is obtained.
With few data sources involved, some simple result backtracking can be performed with structured query language (Structured Query Language, SQL) statements on a traditional relational data basis. However, in practical situations, the number of data sources involved is generally relatively large, and it may be involved that thousands of data sources need to be subjected to various conventional operations or operator processes to obtain the required model data, in this case, the result backtracking of the model data cannot be achieved on the basis of conventional relational data and using SQL statements.
Disclosure of Invention
In view of this, an objective of the embodiments of the present application is to provide a result backtracking method, apparatus, electronic device, and storage medium, so as to implement result backtracking of model data under the condition that data sources are relatively more and data source calculation is complex.
In a first aspect, an embodiment of the present application provides a result backtracking method, where the method includes: receiving a first backtracking request sent by electronic equipment; responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display; receiving a second backtracking request which is sent by the electronic equipment and used for backtracking the data to be backtracked in the whole model data; the second backtracking request comprises an identifier of the data to be backtracked; responding to the second backtracking request, and searching metadata corresponding to the identification on the basis of a first corresponding relation between the prestored identification and metadata of a data source; inputting the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and sending the corresponding data source to the electronic equipment.
Under the condition that more data sources exist and the calculation of the data sources is complex, the result backtracking of the model data cannot be realized by using a traditional relational data basis and utilizing structured query language (Structured Query Language, SQL) sentences, so that in the realization process, the metadata corresponding to the identification is quickly determined based on the corresponding relation between the identification of the data to be backtracked and the metadata of the data sources, and the data sources corresponding to the corresponding metadata are quickly found out from a data storage system by utilizing a pre-trained big data platform.
Based on the first aspect, in one possible design, the second backtracking request further includes: the data to be traced back; before searching for metadata corresponding to the identification, the method further comprises: responding to the second backtracking request, and determining the data type of an algorithm matched with the data to be backtracked from the data types of the predetermined algorithm; wherein sending the corresponding data source to the electronic device comprises: determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
In the result backtracking process, there may be a plurality of data sources corresponding to the data to be backtracked, however, the user may only need some of the data sources, in this case, if all the data sources corresponding to the data to be backtracked are sent to the user, the user may not be able to quickly determine the part of the data sources to be acquired from the received plurality of data sources, so in the implementation process, the data type of the algorithm matching with the data to be backtracked is determined from the data types of the predetermined algorithm, and then the data matching with the data type of the matched algorithm in the corresponding data sources is sent to the electronic device, without sending all the corresponding data sources to the electronic device, thereby improving user experience.
Based on the first aspect, in one possible design, before acquiring the first backtracking request sent by the electronic device, the method further includes: receiving a calculation request for representing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; responding to the calculation request, and based on the name of the corresponding data source, searching metadata corresponding to the name of the corresponding data source from a metadata base; inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtraced based on the corresponding data source and the algorithm; storing the data to be backtraced into the data storage system; the same identification is allocated to the metadata corresponding to the names of the corresponding data sources and the data to be backtraced; and storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
In the implementation process, a large data platform is utilized to search a data source corresponding to the corresponding metadata from the data storage system, the corresponding data source is processed based on the corresponding data source and the algorithm, the data to be traced is output, the complex calculation of a large number of data sources is applicable, the same identification is allocated to the metadata corresponding to the names of the corresponding data sources and the data to be traced, and the same identification is stored in the first corresponding relation of the metadata of the corresponding data sources, so that the association of the corresponding data source and the data to be traced is realized, and the quick realization result tracing based on the identification and the metadata is facilitated.
Based on the first aspect, in one possible design, after storing the data to be backtraced in the data storage system, the method further includes: generating metadata of the data to be traced; storing metadata of the data to be traced back into the metadata base; wherein before sending all pre-stored model data to the electronic device, the method further comprises: responding to the first backtracking request, and acquiring metadata of all model data from the metadata base; and searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
In the implementation process, metadata of the data to be traced is stored, so that the data to be traced can be quickly searched from a data storage system according to the metadata of the data to be traced.
Based on the first aspect, in one possible design, before obtaining the calculation request for characterizing the need to calculate the corresponding data source, the method further includes: acquiring the corresponding data source; storing the corresponding data sources into the data storage system to obtain metadata of the corresponding data sources; wherein, the metadata of the data source comprises the name of the corresponding data source; and storing the metadata of the corresponding data sources into the metadata base.
In the implementation process, the corresponding data sources are stored in the data storage system, and metadata of the corresponding data sources are stored in the metadata base, wherein the metadata of the data sources comprise names of the corresponding data sources, so that metadata of the corresponding data sources can be quickly found out from the metadata base according to the names of the corresponding data sources, and then the corresponding data sources can be quickly found out from the data storage system according to the metadata of the corresponding data sources.
In a second aspect, an embodiment of the present application provides a result backtracking apparatus, including: the first receiving unit is used for receiving a first backtracking request sent by the electronic equipment; the first response unit is used for responding to the first backtracking request and sending all pre-stored model data to the electronic equipment for display; the second receiving unit is used for receiving a second backtracking request which is sent by the electronic equipment and used for characterizing that backtracking of data to be backtracked in the whole model data is needed; the second backtracking request comprises an identifier of the data to be backtracked; the second response unit is used for responding to the second backtracking request and searching metadata corresponding to the identification in a first corresponding relation between the prestored identification and the metadata of the data source; the backtracking unit is used for inputting the corresponding metadata into a pre-trained big data platform so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and the sending unit is used for sending the corresponding data source to the electronic equipment.
Based on the second aspect, in one possible design, the apparatus further comprises: the first determining unit is used for responding to the second backtracking request and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithm; the sending unit is specifically configured to determine, from the corresponding data sources, data that matches the data type of the matched algorithm based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
Based on the second aspect, in one possible design, the apparatus further comprises: a third receiving unit, configured to receive a calculation request for characterizing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; a third response unit, configured to respond to the calculation request, and find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source; the output unit is used for inputting the corresponding metadata and the algorithm into the big data platform so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata and outputs the data to be traced based on the corresponding data source and the algorithm; the first storage unit is used for storing the data to be traced back into the data storage system; the identifier distribution unit is used for distributing the same identifier for the metadata corresponding to the name of the corresponding data source and the data to be traced; and the second storage unit is used for storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
Based on the second aspect, in one possible design, the apparatus further comprises: the generation unit is used for generating metadata of the data to be backtraced; the third storage unit is used for storing the metadata of the data to be traced back into the metadata base; a fourth response unit, configured to obtain metadata of the all model data from the metadata database in response to the first backtracking request; and the searching unit is used for searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
Based on the second aspect, in one possible design, the apparatus further comprises: the acquisition unit is used for acquiring the corresponding data sources; a fourth storage unit, configured to store the corresponding data source into the data storage system, to obtain metadata of the corresponding data source; wherein, the metadata of the data source comprises the name of the corresponding data source; and the fifth storage unit is used for storing the metadata of the corresponding data source into the metadata base.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory connected to the processor, where the memory stores a computer program, and when the computer program is executed by the processor, causes the electronic device to perform the method of the first aspect.
In a fourth aspect, embodiments of the present application provide a storage medium having a computer program stored therein, which when run on a computer causes the computer to perform the method of the first aspect.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the embodiments of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered limiting the scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a result backtracking method provided in an embodiment of the present application.
Fig. 2 is a schematic structural diagram of a result backtracking device according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
Referring to fig. 1, fig. 1 is a flowchart of a result backtracking method according to an embodiment of the present application, where the method includes the steps of: s100, S200, S300, S400, S500, and S600.
S100: and receiving a first backtracking request sent by the electronic equipment.
S200: and responding to the first backtracking request, and sending all prestored model data to the electronic equipment for display.
S300: receiving a second backtracking request which is sent by the electronic equipment and used for backtracking the data to be backtracked in the whole model data; the second backtracking request comprises an identifier of the data to be backtracked.
S400: and responding to the second backtracking request, and searching metadata corresponding to the identification in a first corresponding relation based on the prestored identification and the metadata of the data source.
S500: inputting the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
S600: and sending the corresponding data source to the electronic equipment.
The above method is described in detail below.
As an embodiment, before S100, the method further comprises the steps of: a1, A2, A3, A4, A5 and A6.
As an embodiment, before A1, the method further comprises: a11, a12 and a13.
A11: and acquiring the corresponding data source.
The corresponding data sources may include related data of shopping types, related numbers of travel types, related data of express types, related data of travel types, identity information of key characters, and the like. For example, the travel type related data includes: data of vehicles such as trains, airplanes, buses and the like.
For example, the information on the rides numbered G# # trains at 2019-12-15 for Ma Mou and Zhao Mou is shown below.
ID card name purchase date number state riding date
110101801812 Ma Mou 2019-12-10G# -Z2019-12-15
110103201812 Zhao Mou 2019-12-11G# -Z2019-12-15
For example, the information of the important characters Ma Mou and Zhao Mou is as follows.
Name of identity card
110101801812 and Ma Mou
110103201812 and Zhao Mou
107291023456 and Zhang Mou
The manner of acquiring the corresponding data source may be acquired from a database of a third party storing the corresponding data source, or may be acquired in other manners.
A12: storing the corresponding data sources into the data storage system to obtain metadata of the corresponding data sources; wherein the metadata of the data source comprises the name of the corresponding data source.
Storing the corresponding data sources in the data storage system according to a preset data classification mode, wherein the preset classification mode can be set according to user requirements, for example, classification can be performed according to types of the data sources, travel is classified into one type, shopping is classified into one type, taking a train is classified into one type, taking an airplane is classified into one type and the like, the corresponding data sources can be stored in a form of a table or a file, metadata of the corresponding data sources are generated according to a storage path of the corresponding data sources, and when the corresponding data sources are stored in the form of the table, the metadata of the corresponding data sources comprise: the table names of the corresponding data sources (i.e. the names of the corresponding data sources) are stored, the Chinese meanings of the table names, the field names in the table, the Chinese meanings of the field names, the field types and the like are stored, and different types of data sources are stored in different tables, so that the data sources can be conveniently distinguished, and the management and the searching of the data are conveniently carried out.
In this embodiment, the data storage system may be a distributed file system (Hadoop Distributed File System, HDFS) system to support storage of a large amount of data, and in other embodiments, other types of systems may be used to store the corresponding data sources. It is noted that when the corresponding data source needs to be stored in the HDFS system, HIVE is typically used to map the corresponding data source into a table structure, and metadata of the table structure (i.e., metadata of the corresponding data source) is stored in the MYSQL database. For example, the data sources for riding a train are stored in the form of a table, as shown in table 1.
TABLE 1
For example, the important person information is stored in the form of a table, as shown in table 2.
TABLE 2
A13: and storing the metadata of the corresponding data sources into the metadata base.
Metadata of the corresponding data sources may be stored in the metadata base in the form of tables or files.
After storing the metadata of the corresponding data source to the metadata base, executing step A1.
A1: receiving a calculation request for representing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source.
Wherein the algorithm may be intersection, union, or the like.
In the practical implementation process, A1 may be implemented in a manner that after a user logs in a web page or (Application, APP) through an electronic device, the name of the corresponding data source is selected in a data source input box provided by the web page or APP through the electronic device, the name of the algorithm is selected in an algorithm input box, a custom algorithm (for example, a person with a key person riding on a train more than three times or more is calculated) may also be input in the algorithm input box, after the input of the corresponding data source and the algorithm is completed through the selection and characterization, the electronic device generates and transmits a calculation request for characterizing that calculation needs to be performed on the corresponding data source to a server, and the server receives the calculation request.
As an embodiment, the calculation request may further include: the data types of the data output after the corresponding data sources are calculated can be used for screening the unnecessary data, and user experience is improved.
After receiving the calculation request, the server performs step A2.
A2: and responding to the calculation request, and based on the name of the corresponding data source, searching metadata corresponding to the name of the corresponding data source from a metadata base.
And responding to the calculation request, extracting the name of the corresponding data source and the algorithm from the calculation request, and searching metadata comprising the name of the corresponding data source, namely metadata corresponding to the name of the corresponding data source, from the metadata base based on the name of the corresponding data source.
After the corresponding metadata and the algorithm are acquired, step A3 is performed.
A3: and inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtraced based on the corresponding data source and the algorithm.
The big data platform is a pre-trained big data platform.
The big data platform searches a table or a file storing the corresponding data source from the data storage system based on the storage path in the corresponding metadata and the name of the corresponding data source, extracts the corresponding data source from the table or the file storing the corresponding data source based on the information such as the field name, the field type and the like in the corresponding metadata, and then calculates the corresponding data source by utilizing the algorithm to obtain and output the data to be traced.
For example, the corresponding data sources include: tables 1 and 2; the algorithm is as follows: solving an intersection;
then, by intersecting the data in table 1 and table 2, the data to be traced back are obtained as follows:
name of identity card
110101801812 and Ma Mou
110103201812 and Zhao Mou
After obtaining the data to be backtraced by using the big data platform, the server executes the following steps: A4.
a4: and storing the data to be backtraced into the data storage system.
And (3) the server stores the data to be traced back into the data storage system in the form of a file or a table by utilizing the big data platform, wherein the storage mode of the data to be traced back can refer to the step A12, and the description is omitted.
And after obtaining the metadata corresponding to the names of the corresponding data sources and the data to be backtraced, executing the step A5.
A5: and distributing the same identification for the metadata corresponding to the names of the corresponding data sources and the data to be backtraced.
It will be appreciated that the corresponding data source and the data to be backtraced can be associated by the same identity. It is mentioned that, since the same data source may participate in various calculations with other data sources, and thus the data source corresponds to different data to be traced, the name of the same data source may correspond to multiple identifiers.
A6: and storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
It will be appreciated that the same identifier is added to the metadata of the corresponding data source, so that the metadata of the corresponding data source and the same identifier are stored correspondingly.
In one embodiment, since metadata of the corresponding data source is stored in a metadata base, a correspondence between first metadata of the corresponding data source and the same identifier may be stored, then the first metadata of the metadata may be determined by the same identifier, and then the metadata of the corresponding data source may be found in the metadata base according to the first metadata of the metadata.
As an embodiment, after storing the data to be backtraced into the data storage system, the method further comprises: a41 and a42.
A41: and generating metadata of the data to be traced.
And according to the storage path of the data to be traced, storing the table names, the field names and the like used by the data to be traced to generate metadata of the data to be traced.
A42: and storing the metadata of the data to be traced back into the metadata base.
And storing the metadata of the data to be traced back into the metadata base in the form of a table or a file.
After storing the data to be backtraced, step S100 is performed.
S100: and receiving a first backtracking request sent by the electronic equipment.
In an actual implementation process, S100 may be implemented in a manner that after a user logs in a web page or an APP through an electronic device, the electronic device generates and sends a first backtracking request to a server after selecting a function option indicating that result backtracking is required in the web page or the APP through the electronic device, and the server receives the first backtracking request.
After receiving the first backtracking request, the server performs step S200.
S200: and responding to the first backtracking request, and transmitting all pre-stored model data to the electronic equipment.
And responding to the first backtracking request, acquiring all the model data according to a predetermined storage path, transmitting all the model data to the electronic equipment according to the IP address of the electronic equipment acquired in advance, and displaying the model data on an interface of the electronic equipment in a paging mode.
It will be appreciated that all the model data may be stored in one file or table, or may be stored in a plurality of files or tables, and then all the model data stored in the files or tables are acquired according to the storage paths of the files.
After the entire model data is transmitted to the electronic device, step S300 is performed.
S300: receiving a second backtracking request which is sent by the electronic equipment and used for backtracking the data to be backtracked in the whole model data; the second backtracking request comprises an identifier of the data to be backtracked.
In an actual implementation process, S300 may be implemented in a manner that after the electronic device receives the all model data, after a user selects data to be traced in the all model data through the electronic device, each piece of data to be traced is pre-allocated with an identifier, so that the electronic device can generate and send, according to the identifier of the data to be traced, the second tracing request for tracing the data to be traced in the all model data, where the representation needs to be traced; the second backtracking request comprises an identifier of the data to be backtracked.
After receiving the second backtracking request, the server performs step S400.
S400: and responding to the second backtracking request, and searching metadata corresponding to the identification in a first corresponding relation based on the prestored identification and the metadata of the data source.
And responding to the second backtracking request, extracting the identification of the data to be backtracked from the second backtracking request, and then searching the metadata corresponding to the identification of the data to be backtracked from the first relation of the pre-stored identification and the metadata of the data source according to the identification of the data to be backtracked.
After the corresponding metadata is acquired, step S500 is performed.
S500: inputting the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
In the actual implementation process, after the pre-trained big data platform acquires the corresponding metadata, according to the storage path in the corresponding metadata and the name of the corresponding data source, a table or a file for storing the corresponding data source is searched from the data storage system.
As one embodiment, after searching the table or the file storing the corresponding data source, the corresponding data source is extracted from the table or the file storing the corresponding data source based on the information such as the field name, the field type and the like in the corresponding metadata.
S600: and sending the corresponding data source to the electronic equipment.
And sending the table or the file storing the corresponding data source to the electronic equipment.
As an embodiment, the corresponding data source is directly transmitted to the electronic device.
For example, when the data to be traced is the name Ma Mou, since the data source correlated with the data to be traced Ma Mou includes the data of table 1 and table 2, the data of table 1 and table 2 are transmitted to the electronic device.
As an embodiment, the second backtracking request further includes: the data to be traced back; prior to S400, the method further comprises:
and responding to the second backtracking request, and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithm.
And responding to the second backtracking request, extracting the data to be backtracked from the second backtracking request, then determining the field type of the field where the data to be backtracked is located, namely the data type of the data to be backtracked, comparing the data type of the data to be backtracked with the data types of a plurality of predetermined algorithms one by one, and determining that the data to be backtracked is matched with the data type of one algorithm in the plurality of predetermined algorithms when the data type of the data to be backtracked is determined to be identical with the data type of the algorithm. The manner of determining the field type is well known in the art, and therefore, will not be described herein.
For example, when the field type of the field where the data to be traced is located is an identification card number, the data type of the data to be traced is an identification card number type, and then the algorithm matched with the data to be traced is an identification card number algorithm.
Wherein S600 includes:
and determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm.
And determining the field type matched with the data type of the matched algorithm from a table or a file storing the corresponding data source based on the data type of the matched algorithm, and then determining the data with the field type being the matched field type from the table or the file of the corresponding data source.
And sending the matched data to the electronic equipment.
And sending the matched data to the electronic equipment in the form of a table and a file.
As an embodiment, before S200, the method further includes: b1 and B2.
B1: and responding to the first backtracking request, and acquiring metadata of all the model data from the metadata base.
After receiving the first backtracking request, responding to the first backtracking request, and acquiring metadata of all model data from the metadata base based on a predetermined storage path.
After the metadata of the entire model data is acquired, step B2 is performed.
B2: and searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
In the specific embodiment of B2, reference may be made to step S500, and thus, details are not repeated herein.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a result backtracking device according to an embodiment of the present application, where the device includes:
the first receiving unit 410 is configured to receive a first backtracking request sent by the electronic device.
And the first response unit 420 is configured to send all pre-stored model data to the electronic device for display in response to the first backtracking request.
A second receiving unit 430, configured to receive a second backtracking request sent by the electronic device, where the second backtracking request represents that backtracking needs to be performed on data to be backtracked in the all model data; the second backtracking request comprises an identifier of the data to be backtracked.
And the second response unit 440 is configured to respond to the second backtracking request, and search metadata corresponding to the identifier in a first correspondence relationship based on the pre-stored identifier and metadata of the data source.
And the backtracking unit 450 is configured to input the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
And a sending unit 460, configured to send the corresponding data source to the electronic device.
As an embodiment, the apparatus further comprises: the first determining unit is used for responding to the second backtracking request and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithm; the sending unit 460 is specifically configured to determine, from the corresponding data sources, data that matches the data type of the matching algorithm based on the data type of the matching algorithm; and sending the matched data to the electronic equipment.
As an embodiment, the apparatus further comprises: a third receiving unit, configured to receive a calculation request for characterizing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; a third response unit, configured to respond to the calculation request, and find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source; the output unit is used for inputting the corresponding metadata and the algorithm into the big data platform so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata and outputs the data to be traced based on the corresponding data source and the algorithm; the first storage unit is used for storing the data to be traced back into the data storage system; the identifier distribution unit is used for distributing the same identifier for the metadata corresponding to the name of the corresponding data source and the data to be traced; and the second storage unit is used for storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
As an embodiment, the apparatus further comprises: the generation unit is used for generating metadata of the data to be backtraced; the third storage unit is used for storing the metadata of the data to be traced back into the metadata base; a fourth response unit, configured to obtain metadata of the all model data from the metadata database in response to the first backtracking request; and the searching unit is used for searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
As an embodiment, the apparatus further comprises: the acquisition unit is used for acquiring the corresponding data sources; a fourth storage unit, configured to store the corresponding data source into the data storage system, to obtain metadata of the corresponding data source; wherein, the metadata of the data source comprises the name of the corresponding data source; and the fifth storage unit is used for storing the metadata of the corresponding data source into the metadata base.
For the process of implementing the respective functions by the functional units in this embodiment, please refer to the content described in the embodiment shown in fig. 1, which is not described herein again.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, where in the embodiment of the present application, the electronic device is a server mentioned in an embodiment of a result backtracking method, and the electronic device may be a tablet computer, a smart phone, a personal digital assistant (personal digital assistant, PDA), or the like.
The electronic device may include: memory 102, process 101, communication interface 103, and a communication bus for enabling connected communication of these components.
The Memory 102 is used for storing all model data, the first correspondence, and various data such as computer program instructions corresponding to the result backtracking method and apparatus provided in the embodiments of the present application, where the Memory 102 may be, but is not limited to, a random access Memory (RandomAccessMemory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), and the like.
The processor 101 is configured to execute the result backtracking method provided in the embodiment of the present application when reading and executing the computer program instructions corresponding to the result backtracking method stored in the memory.
The processor 101 may be an integrated circuit chip with signal processing capability. The processor 101 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), discrete gate or transistor logic, discrete hardware components.
And the communication interface 103 is configured to receive the first backtracking request, and send the corresponding data source to a device that sends the first backtracking request.
Furthermore, the embodiment of the present application provides a storage medium, in which a computer program is stored, which when executed on a computer, causes the computer to perform the method provided in any one of the embodiments of the present application.
In summary, the result backtracking method, the device, the electronic equipment and the storage medium provided by the embodiments of the present application cannot implement result backtracking of model data based on conventional relational data base and using SQL statements under the condition that data sources are relatively many and calculation of the data sources is relatively complex, so that metadata corresponding to an identifier is quickly determined based on a correspondence between the identifier of the data to be backtracked and metadata of the data sources, and a data source corresponding to the corresponding metadata is quickly found out from a data storage system by using a pre-trained big data platform.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners as well. The apparatus embodiments described above are merely illustrative, for example, flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based devices which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (8)
1. A method of backtracking results, the method comprising:
receiving a first backtracking request sent by electronic equipment;
responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display;
receiving a second backtracking request which is sent by the electronic equipment and used for backtracking the data to be backtracked in the whole model data; the second backtracking request comprises an identifier of the data to be backtracked;
responding to the second backtracking request, and searching metadata corresponding to the identification on the basis of a first corresponding relation between the prestored identification and metadata of a data source;
Inputting the corresponding metadata into a pre-trained big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata;
transmitting the corresponding data source to the electronic equipment;
before receiving the first backtracking request sent by the electronic device, the method further includes:
receiving a calculation request for representing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source;
responding to the calculation request, and based on the name of the corresponding data source, searching metadata corresponding to the name of the corresponding data source from a metadata base;
inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtraced based on the corresponding data source and the algorithm;
storing the data to be backtraced into the data storage system;
The same identification is allocated to the metadata corresponding to the names of the corresponding data sources and the data to be backtraced;
and storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
2. The method of claim 1, wherein the second backtracking request further comprises: the data to be traced back; before searching for metadata corresponding to the identification, the method further comprises:
responding to the second backtracking request, and determining the data type of an algorithm matched with the data to be backtracked from the data types of the predetermined algorithm;
wherein sending the corresponding data source to the electronic device comprises:
determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm;
and sending the matched data to the electronic equipment.
3. The method of claim 1, wherein after storing the data to be backtraced in the data storage system, the method further comprises:
generating metadata of the data to be traced;
storing metadata of the data to be traced back into the metadata base;
Wherein before sending all pre-stored model data to the electronic device, the method further comprises:
responding to the first backtracking request, and acquiring metadata of all model data from the metadata base;
and searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
4. The method of claim 1, wherein prior to obtaining a calculation request for characterizing a need to calculate the corresponding data source, the method further comprises:
acquiring the corresponding data source;
storing the corresponding data sources into the data storage system to obtain metadata of the corresponding data sources; wherein, the metadata of the data source comprises the name of the corresponding data source;
and storing the metadata of the corresponding data sources into the metadata base.
5. A result backtracking apparatus, the apparatus comprising:
the first receiving unit is used for receiving a first backtracking request sent by the electronic equipment;
the first response unit is used for responding to the first backtracking request and sending all pre-stored model data to the electronic equipment for display;
The second receiving unit is used for receiving a second backtracking request which is sent by the electronic equipment and used for characterizing that backtracking of data to be backtracked in the whole model data is needed; the second backtracking request comprises an identifier of the data to be backtracked;
the second response unit is used for responding to the second backtracking request and searching metadata corresponding to the identification in a first corresponding relation between the prestored identification and the metadata of the data source;
the backtracking unit is used for inputting the corresponding metadata into a pre-trained big data platform so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata;
a sending unit, configured to send the corresponding data source to the electronic device;
a third receiving unit, configured to receive a calculation request for characterizing that calculation is required for the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source;
a third response unit, configured to respond to the calculation request, and find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source;
The output unit is used for inputting the corresponding metadata and the algorithm into the big data platform so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata and outputs the data to be traced based on the corresponding data source and the algorithm;
the first storage unit is used for storing the data to be traced back into the data storage system;
the identifier distribution unit is used for distributing the same identifier for the metadata corresponding to the name of the corresponding data source and the data to be traced;
and the second storage unit is used for storing the same identification and the first corresponding relation of the metadata of the corresponding data source.
6. The apparatus of claim 5, wherein the apparatus further comprises:
the first determining unit is used for responding to the second backtracking request and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithm;
the sending unit is specifically configured to determine, from the corresponding data sources, data that matches the data type of the matched algorithm based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
7. An electronic device comprising a processor and a memory coupled to the processor, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method of any of claims 1-4.
8. A storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387737.2A CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387737.2A CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111125472A CN111125472A (en) | 2020-05-08 |
| CN111125472B true CN111125472B (en) | 2023-05-02 |
Family
ID=70505999
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911387737.2A Active CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111125472B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364096B (en) * | 2021-01-13 | 2021-07-06 | 天云融创数据科技（北京）有限公司 | High-dimensional metadata management system and method based on complex network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| CN110032575A (en) * | 2019-04-15 | 2019-07-19 | 网易（杭州）网络有限公司 | Data query method, apparatus, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108733316B (en) * | 2017-04-17 | 2021-08-31 | 伊姆西Ip控股有限责任公司 | Method and manager for managing storage system |
-
2019
- 2019-12-27 CN CN201911387737.2A patent/CN111125472B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| CN110032575A (en) * | 2019-04-15 | 2019-07-19 | 网易（杭州）网络有限公司 | Data query method, apparatus, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111125472A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ferreira et al. | Emergent: a novel data-set for stance classification | |
| JP5575902B2 (en) | Information retrieval based on query semantic patterns | |
| CN106934254B (en) | Analysis method and device for open source license | |
| US20150186787A1 (en) | Cloud-based plagiarism detection system | |
| US20220244988A1 (en) | Data shards for distributed processing | |
| CN110929125A (en) | Search recall method, apparatus, device and storage medium thereof | |
| CN105930409A (en) | Data verification method and device based on dynamic generation rule | |
| CN106815265B (en) | Method and device for searching referee document | |
| WO2020207084A1 (en) | Payment complaint method, device, server, and readable storage medium | |
| CN112883190A (en) | Text classification method and device, electronic equipment and storage medium | |
| CN111666940A (en) | Chat screenshot content processing method and device, electronic equipment and readable storage medium | |
| CN112231417A (en) | Data classification method and device, electronic equipment and storage medium | |
| CN111125472B (en) | Result backtracking method and device, electronic equipment and storage medium | |
| CN112559526A (en) | Data table export method and device, computer equipment and storage medium | |
| CN111445319A (en) | Voucher generation method and device, computer equipment and storage medium | |
| WO2020155814A1 (en) | Damage determination method and apparatus for maintenance object, and an electronic device | |
| CN111858834B (en) | Case dispute focus determining method, device, equipment and medium based on AI | |
| CN111639903A (en) | Review processing method for architecture change and related equipment | |
| CN115203339A (en) | Multi-data source integration method and device, computer equipment and storage medium | |
| CN111859985B (en) | AI customer service model test method and device, electronic equipment and storage medium | |
| CN113901075A (en) | Method and device for generating SQL (structured query language) statement, computer equipment and storage medium | |
| CN113821594A (en) | Text processing method and device and readable storage medium | |
| CN112347201A (en) | Information processing method, information processing device and terminal equipment | |
| CN112328752A (en) | Course recommendation method and device based on search content, computer equipment and medium | |
| CN111125185A (en) | Data processing method, device, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |