CN111125472A - Result backtracking method and device, electronic equipment and storage medium - Google Patents
Result backtracking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111125472A CN111125472A CN201911387737.2A CN201911387737A CN111125472A CN 111125472 A CN111125472 A CN 111125472A CN 201911387737 A CN201911387737 A CN 201911387737A CN 111125472 A CN111125472 A CN 111125472A
- Authority
- CN
- China
- Prior art keywords
- data
- metadata
- data source
- backtracking
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a result backtracking method, a result backtracking device, an electronic device and a storage medium, wherein the method comprises the following steps: receiving a first backtracking request sent by electronic equipment; responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display; receiving a second backtracking request which is sent by the electronic equipment and represents that backtracking needs to be carried out on data to be backtracked in all model data; the second backtracking request comprises an identifier of data to be backtracked; responding to a second backtracking request, and searching metadata corresponding to the identifier based on a first corresponding relation between the pre-stored identifier and metadata of the data source; inputting the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and sending the corresponding data source to the electronic equipment so as to realize result backtracking when the data sources are more and the calculation is complex.
Description
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a result backtracking method and apparatus, an electronic device, and a storage medium.
Background
In the data modeling system, model data is obtained by processing one or more data sources by using certain algorithms or operations, however, the result backtracking refers to backtracking the data source associated with the model data according to the obtained model data after obtaining the model data.
Under the condition of relatively few involved data sources, some simple result backtracking can be performed by using Structured Query Language (SQL) statements on the basis of traditional relational data. However, in an actual situation, the number of related data sources is usually large, and it may be that thousands of data sources need to undergo various conventional operations or operator processing to obtain the required model data, and in such a situation, the result backtracking of the model data cannot be realized by using SQL statements based on the conventional relational data.
Content of application
In view of this, an object of the embodiments of the present application is to provide a result backtracking method, apparatus, electronic device and storage medium, so as to implement result backtracking of model data under the conditions that data sources are more and data source computation is complex.
In a first aspect, an embodiment of the present application provides a result backtracking method, where the method includes: receiving a first backtracking request sent by electronic equipment; responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display; receiving a second backtracking request which is sent by the electronic equipment and represents that backtracking needs to be carried out on data to be backtracked in all the model data; the second backtracking request comprises an identifier of the data to be backtracked; responding to the second backtracking request, and searching metadata corresponding to the identifier based on a first corresponding relation between the pre-stored identifier and metadata of the data source; inputting the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and sending the corresponding data source to the electronic equipment.
Under the condition that data sources are more and the calculation of the data sources is more complex, the result backtracking of model data cannot be realized by using a traditional relational data base and Structured Query Language (SQL) statements, so that in the implementation process, the metadata corresponding to the identifier is quickly determined based on the corresponding relationship between the identifier of the data to be backtracked and the metadata of the data sources, and the data sources corresponding to the corresponding metadata are quickly found out from a data storage system by using a pre-trained big data platform.
Based on the first aspect, in a possible design, the second backtracking request further includes: the data to be backtracked; before searching the metadata corresponding to the identifier, the method further comprises: responding to the second backtracking request, and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithms; wherein sending the corresponding data source to the electronic device includes: determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
In the process of result backtracking, a plurality of data sources corresponding to data to be backtracked may exist, however, a user may only need a part of the data sources, in this case, if all the data sources corresponding to the data to be backtracked are sent to the user, the user cannot quickly determine the part of the data sources to be acquired from the received data sources, and therefore, in the implementation process, the data type of the algorithm matched with the data to be backtracked is determined from the data types of the predetermined algorithm, and then the data matched with the data type of the matched algorithm in the corresponding data sources is sent to the electronic device, so that all the corresponding data sources are not needed to be sent to the electronic device, and the user experience is improved.
Based on the first aspect, in a possible design, before obtaining the first trace-back request sent by the electronic device, the method further includes: receiving a calculation request for representing that calculation needs to be carried out on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; in response to the calculation request, searching metadata corresponding to the name of the corresponding data source from a metadata base based on the name of the corresponding data source; inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm; storing the data to be backtracked into the data storage system; distributing the same identification for the metadata corresponding to the name of the corresponding data source and the data to be backtracked; storing the same identification with the first correspondence of metadata of the corresponding data source.
In the implementation process, a big data platform is used for searching a data source corresponding to the corresponding metadata from the data storage system, the corresponding data source is processed based on the corresponding data source and the algorithm, the data to be backtracked is output, the method is suitable for complex calculation of a large number of data sources, the same identification is distributed to the metadata corresponding to the name of the corresponding data source and the data to be backtracked, the first corresponding relation between the same identification and the metadata of the corresponding data source is stored, the association between the corresponding data source and the data to be backtracked is realized, and the rapid implementation result backtracking based on the identification and the metadata is facilitated.
Based on the first aspect, in a possible design, after the data to be traced is stored in the data storage system, the method further includes: generating metadata of the data to be backtracked; storing the metadata of the data to be backtracked into the metadata database; before all the pre-stored model data are sent to the electronic device, the method further comprises: responding to the first backtracking request, and acquiring metadata of all model data from the metadata base; and searching all model data corresponding to the metadata of all model data from the data storage system based on the metadata of all model data.
In the implementation process, the metadata of the data to be backtracked is stored, so that the data to be backtracked can be rapidly found out from a data storage system according to the metadata of the data to be backtracked.
Based on the first aspect, in a possible design, before obtaining the computation request for characterizing the computation required for the corresponding data source, the method further includes: acquiring the corresponding data source; storing the corresponding data source into the data storage system to obtain metadata of the corresponding data source; wherein the metadata of the data source comprises the name of the corresponding data source; and storing the metadata of the corresponding data source into the metadata database.
In the implementation process, the corresponding data source is stored in a data storage system, and the metadata of the corresponding data source is stored in a metadata database, wherein the metadata of the data source includes the name of the corresponding data source, so that the metadata of the corresponding data source can be quickly found from the metadata database subsequently according to the name of the corresponding data source, and then the corresponding data source can be quickly found from the data storage system according to the metadata of the corresponding data source.
In a second aspect, an embodiment of the present application provides a result backtracking apparatus, including: the first receiving unit is used for receiving a first backtracking request sent by the electronic equipment; the first response unit is used for responding to the first backtracking request and sending all pre-stored model data to the electronic equipment for display; a second receiving unit, configured to receive a second trace-back request that represents that data to be traced back in all the model data needs to be traced back, where the second trace-back request is sent by the electronic device; the second backtracking request comprises an identifier of the data to be backtracked; a second response unit, configured to respond to the second backtracking request, and search for metadata corresponding to the identifier based on a first correspondence between a pre-stored identifier and metadata of a data source; the backtracking unit is used for inputting the corresponding metadata into a big data platform trained in advance so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata; and the sending unit is used for sending the corresponding data source to the electronic equipment.
Based on the second aspect, in one possible design, the apparatus further includes: a first determining unit, configured to determine, in response to the second backtracking request, a data type of an algorithm that matches the data to be backtracked from data types of predetermined algorithms; the sending unit is specifically configured to determine, from the corresponding data source, data that matches the data type of the matched algorithm based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
Based on the second aspect, in one possible design, the apparatus further includes: a third receiving unit, configured to receive a calculation request for representing that calculation needs to be performed on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; a third response unit, configured to, in response to the calculation request, find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source; the output unit is used for inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm; the first storage unit is used for storing the data to be backtracked into the data storage system; the identification distribution unit is used for distributing the same identification for the metadata corresponding to the name of the corresponding data source and the data to be backtracked; a second storage unit, configured to store the same identifier and the first corresponding relationship of the metadata of the corresponding data source.
Based on the second aspect, in one possible design, the apparatus further includes: the generating unit is used for generating metadata of the data to be backtracked; the third storage unit is used for storing the metadata of the data to be backtracked into the metadata database; a fourth response unit, configured to respond to the first backtracking request, and obtain metadata of all the model data from the metadata database; and the searching unit is used for searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
Based on the second aspect, in one possible design, the apparatus further includes: an obtaining unit, configured to obtain the corresponding data source; a fourth storage unit, configured to store the corresponding data source in the data storage system, so as to obtain metadata of the corresponding data source; wherein the metadata of the data source comprises the name of the corresponding data source; and the fifth storage unit is used for storing the metadata of the corresponding data source into the metadata database.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory connected to the processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device is caused to perform the method of the first aspect.
In a fourth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the method of the first aspect.
Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a flowchart of a result backtracking method according to an embodiment of the present disclosure.
Fig. 2 is a schematic structural diagram of a result backtracking apparatus according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solution in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Referring to fig. 1, fig. 1 is a flowchart of a result backtracking method according to an embodiment of the present application, where the method includes: s100, S200, S300, S400, S500, and S600.
S100: receiving a first backtracking request sent by the electronic equipment.
S200: and responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display.
S300: receiving a second backtracking request which is sent by the electronic equipment and represents that backtracking needs to be carried out on data to be backtracked in all the model data; and the second backtracking request comprises the identifier of the data to be backtracked.
S400: and responding to the second backtracking request, and searching metadata corresponding to the identifier based on a first corresponding relation between the pre-stored identifier and metadata of the data source.
S500: inputting the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
S600: and sending the corresponding data source to the electronic equipment.
The above method is described in detail below.
As an embodiment, before S100, the method further includes the step of: a1, a2, A3, a4, a5, and a 6.
As an embodiment, before a1, the method further comprises: a11, a12 and a 13.
A11: and acquiring the corresponding data source.
The corresponding data source may include data related to shopping types, data related to travel types, data related to express delivery types, data related to travel types, identity information of key persons, and the like. For example, the relevant data of the travel types includes: data for vehicles such as trains, planes, and buses.
For example, the information for a ma and a zhao train with the number G53 in 2019-12-15 is as follows.
For example, the information of the emphasized characters ma chi and zhao chi is as follows.
The manner of acquiring the corresponding data source may be acquired from a database of a third party that stores the corresponding data source, or may be acquired in other manners.
A12: storing the corresponding data source into the data storage system to obtain metadata of the corresponding data source; wherein the metadata of the data source includes the name of the corresponding data source.
The corresponding data sources are classified and stored in the data storage system according to a preset data classification mode, wherein the preset classification mode can be set according to user requirements, for example, classification can be performed according to types of the data sources, classification of travel, classification of shopping, classification of train riding, classification of airplane riding and the like, wherein the corresponding data sources can be stored in a table form or a file form, metadata of the corresponding data sources are generated according to storage paths of the corresponding data sources, and when the corresponding data sources are stored in the table form, the metadata of the corresponding data sources include: the table name of the corresponding data source (namely the name of the corresponding data source), the storage path, the Chinese meaning of the table name, the field name in the table, the Chinese meaning and the field type of the field name, and the like are stored, and the data sources of different classes are stored in different tables so as to be convenient for distinguishing the data sources and managing and searching data.
In this embodiment, the data storage System may be a distributed File System (HDFS) System to support storage of a large amount of data, and in other embodiments, other types of systems may also be used to store the corresponding data source. As a value, when the corresponding data source needs to be stored in the HDFS system, the corresponding data source is mapped into a table structure, and metadata of the table structure (i.e. metadata of the corresponding data source) is stored in the MYSQL database. For example, the data sources for riding in the train are stored in a table format, as shown in table 1.
| idno | name | purchase_date | train_no | train_status | train_date |
| 11010180181234173X | Horse's horn | 2019-12-10 | G53 | Z | 2019-12-15 |
| 110103201812345681 | Zhao (a certain thing) | 2019-12-11 | G53 | Z | 2019-12-15 |
TABLE 1
For example, the highlight personal information is stored in the form of a table, as shown in table 2.
| idno | name |
| 11010180181234173X | Horse's horn |
| 110103201812345681 | Zhao (a certain thing) |
| 10729102345620136X | Zhang-a |
TABLE 2
A13: and storing the metadata of the corresponding data source into the metadata database.
The metadata of the corresponding data source may be stored in the metadata repository in the form of a table or a file.
After storing the metadata of the corresponding data source in the metadata repository, step a1 is performed.
A1: receiving a calculation request for representing that calculation needs to be carried out on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source.
The algorithm may be to find an intersection, a union, and the like.
In an actual implementation process, a1 may be implemented in such a manner that, after a user logs in a web page or (Application, APP) through an electronic device, the user selects a name of the corresponding data source in a data source input box provided by the web page or APP and selects a name of the algorithm in the algorithm input box, or a customized algorithm (for example, a person who calculates a key person to ride in a train more than three times or more) may be input in the algorithm input box, and after the input of the corresponding data source and the algorithm is completed by selecting a representation, the electronic device generates and sends a calculation request representing that calculation needs to be performed on the corresponding data source to a server, and the server receives the calculation request.
As an embodiment, the computing request may further include: the data type of the data output after the corresponding data source is calculated can be screened by the method, so that the user experience is improved.
The server performs step a2 after receiving the calculation request.
A2: and in response to the calculation request, searching metadata corresponding to the name of the corresponding data source from a metadata base based on the name of the corresponding data source.
And responding to the calculation request, extracting the name of the corresponding data source and the algorithm from the calculation request, and searching metadata comprising the name of the corresponding data source, namely the metadata corresponding to the name of the corresponding data source from the metadata base based on the name of the corresponding data source.
After the corresponding metadata and the algorithm are obtained, step a3 is performed.
A3: inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm.
The big data platform is a big data platform which is trained in advance.
And the big data platform searches a table or a file for storing the corresponding data source from the data storage system based on the storage path in the corresponding metadata and the name of the corresponding data source, extracts the corresponding data source from the table or the file for storing the corresponding data source based on information such as a field name, a field type and the like in the corresponding metadata, calculates the corresponding data source by using the algorithm, and obtains and outputs the data to be traced.
For example, the corresponding data sources include: tables 1 and 2; the algorithm is as follows: solving an intersection;
then, by performing intersection on the data in table 1 and table 2, the obtained data to be backtracked is:
identity card name
11010180181234173X Ma Zhi
110103201812345681 Zhao
After the server obtains the data to be backtracked by using the big data platform, executing the following steps: A4.
a4: and storing the data to be backtracked into the data storage system.
The server stores the data to be backtracked to the data storage system in a file or table form by using the big data platform, wherein the storage manner of the data to be backtracked may refer to step a12, which is not described herein again.
After the metadata corresponding to the name of the corresponding data source and the data to be traced are acquired, step a5 is executed.
A5: and distributing the same identifier for the metadata corresponding to the name of the corresponding data source and the data to be traced.
It is understood that the corresponding data source and the data to be traced can be associated by the same identifier. The data source may participate in various different calculations with other data sources, so that the data source corresponds to different data to be traced back, and therefore, the name of the same data source may correspond to multiple identifiers.
A6: storing the same identification with the first correspondence of metadata of the corresponding data source.
It is understood that the same identifier is added to the metadata of the corresponding data source, so that the metadata of the corresponding data source and the same identifier are stored correspondingly.
As an implementation manner, since the metadata of the corresponding data source is stored in the metadata database, the corresponding relationship between the first metadata of the corresponding data source and the same identifier may be stored, then the first metadata of the metadata is determined by the same identifier, and then the metadata of the corresponding data source is found in the metadata database according to the first metadata of the metadata.
As an embodiment, after storing the data to be traced back into the data storage system, the method further includes: a41 and a 42.
A41: and generating metadata of the data to be backtracked.
And according to the storage path of the data to be backtracked, storing the table name, the field name and the like used by the data to be backtracked to generate the metadata of the data to be backtracked.
A42: and storing the metadata of the data to be backtracked into the metadata database.
And storing the metadata of the data to be backtracked into the metadata database in a form of a table or a file.
After the data to be traced is stored, step S100 is performed.
S100: receiving a first backtracking request sent by the electronic equipment.
In an actual implementation process, S100 may be implemented in the following manner, after a user may log in a webpage or APP through an electronic device, and after a functional option that needs result backtracking is selected and characterized in the webpage or APP through the electronic device, the electronic device generates and sends a first backtracking request to be characterized to a server, and the server receives the first backtracking request.
After receiving the first trace-back request, the server executes step S200.
S200: and responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment.
And responding to the first backtracking request, acquiring all the model data according to a predetermined storage path, sending all the model data to the electronic equipment according to the pre-acquired IP address of the electronic equipment, and displaying the model data on an interface of the electronic equipment in a paging mode.
It is understood that all the model data may be stored in one file or table, or in multiple files or tables, and then all the model data stored in the file or table is obtained according to the storage path of the file.
After transmitting the entire model data to the electronic device, step S300 is performed.
S300: receiving a second backtracking request which is sent by the electronic equipment and represents that backtracking needs to be carried out on data to be backtracked in all the model data; and the second backtracking request comprises the identifier of the data to be backtracked.
In an actual implementation process, step S300 may be implemented in such a manner that after the electronic device receives all the model data, and after a user selects data to be traced in all the model data through the electronic device, because each data to be traced is pre-assigned with an identifier, the electronic device can generate and send a second trace request representing that the data to be traced in all the model data needs to be traced according to the identifier of the data to be traced; and the second backtracking request comprises the identifier of the data to be backtracked.
After receiving the second backtracking request, the server performs step S400.
S400: and responding to the second backtracking request, and searching metadata corresponding to the identifier based on a first corresponding relation between the pre-stored identifier and metadata of the data source.
Responding to the second backtracking request, extracting the identifier of the data to be backtracked from the second backtracking request, and then searching the metadata corresponding to the identifier of the data to be backtracked from the pre-stored identifier and the first relation of the metadata of the data source according to the identifier of the data to be backtracked.
After the corresponding metadata is acquired, step S500 is performed.
S500: inputting the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
In an actual implementation process, after the pre-trained big data platform acquires the corresponding metadata, according to a storage path in the corresponding metadata and the name of the corresponding data source, a table or a file storing the corresponding data source is found from the data storage system.
As an implementation manner, after finding the table or the file storing the corresponding data source, the corresponding data source is extracted from the table or the file storing the corresponding data source based on the field name, the field type, and other information in the corresponding metadata.
S600: and sending the corresponding data source to the electronic equipment.
And sending the table or the file storing the corresponding data source to the electronic equipment.
As an embodiment, the corresponding data source is directly sent to the electronic device.
For example, when the data to be backtracked is named marten, the data in table 1 and table 2 are sent to the electronic device because the data source correlated with the data to be backtracked includes the data in table 1 and table 2.
As an embodiment, the second backtracking request further includes: the data to be backtracked; before S400, the method further comprises:
and responding to the second backtracking request, and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithms.
Responding to the second backtracking request, extracting the data to be backtracked from the second backtracking request, then determining the field type of the field where the data to be backtracked is located, namely the data type of the data to be backtracked, comparing the data type of the data to be backtracked with the data types of multiple predetermined algorithms one by one, and determining that the data to be backtracked is matched with the data type of the algorithm when the data type of the data to be backtracked is determined to be the same as the data type of one of the multiple predetermined algorithms. The manner of determining the field type is well known in the art, and therefore, will not be described herein.
For example, when the field type of the field where the data to be traced is located is an identity card number, the data type of the data to be traced is an identity number type, and then, the algorithm matched with the data to be traced is an identity card number algorithm.
Wherein, S600 includes:
and determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm.
And determining a field type matched with the data type of the matched algorithm from the table or the file storing the corresponding data source based on the data type of the matched algorithm, and then determining data with the field type matched with the data type from the table or the file of the corresponding data source.
And sending the matched data to the electronic equipment.
And sending the matched data to the electronic equipment in a form of a table or a file.
As an embodiment, before S200, the method further includes: b1 and B2.
B1: and responding to the first backtracking request, and acquiring metadata of all model data from the metadata database.
And after receiving the first backtracking request, responding to the first backtracking request, and acquiring metadata of all model data from the metadata base based on a predetermined storage path.
After the metadata of all the model data is acquired, step B2 is executed.
B2: and searching all model data corresponding to the metadata of all model data from the data storage system based on the metadata of all model data.
The step S500 may be referred to for the specific implementation of B2, and therefore, the detailed description thereof is omitted here.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a result backtracking apparatus according to an embodiment of the present application, where the apparatus includes:
the first receiving unit 410 is configured to receive a first trace back request sent by an electronic device.
A first responding unit 420, configured to respond to the first backtracking request, and send all pre-stored model data to the electronic device for display.
A second receiving unit 430, configured to receive a second trace-back request that represents that data to be traced back in all the model data needs to be traced back, where the second trace-back request is sent by the electronic device; and the second backtracking request comprises the identifier of the data to be backtracked.
A second responding unit 440, configured to respond to the second backtracking request, and search for metadata corresponding to the identifier based on a first correspondence between a pre-stored identifier and metadata of the data source.
The backtracking unit 450 is configured to input the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata.
A sending unit 460, configured to send the corresponding data source to the electronic device.
As an embodiment, the apparatus further comprises: a first determining unit, configured to determine, in response to the second backtracking request, a data type of an algorithm that matches the data to be backtracked from data types of predetermined algorithms; the sending unit 460 is specifically configured to determine, based on the data type of the matched algorithm, data that matches the data type of the matched algorithm from the corresponding data source; and sending the matched data to the electronic equipment.
As an embodiment, the apparatus further comprises: a third receiving unit, configured to receive a calculation request for representing that calculation needs to be performed on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source; a third response unit, configured to, in response to the calculation request, find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source; the output unit is used for inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm; the first storage unit is used for storing the data to be backtracked into the data storage system; the identification distribution unit is used for distributing the same identification for the metadata corresponding to the name of the corresponding data source and the data to be backtracked; a second storage unit, configured to store the same identifier and the first corresponding relationship of the metadata of the corresponding data source.
As an embodiment, the apparatus further comprises: the generating unit is used for generating metadata of the data to be backtracked; the third storage unit is used for storing the metadata of the data to be backtracked into the metadata database; a fourth response unit, configured to respond to the first backtracking request, and obtain metadata of all the model data from the metadata database; and the searching unit is used for searching all the model data corresponding to the metadata of all the model data from the data storage system based on the metadata of all the model data.
As an embodiment, the apparatus further comprises: an obtaining unit, configured to obtain the corresponding data source; a fourth storage unit, configured to store the corresponding data source in the data storage system, so as to obtain metadata of the corresponding data source; wherein the metadata of the data source comprises the name of the corresponding data source; and the fifth storage unit is used for storing the metadata of the corresponding data source into the metadata database.
For the process of implementing each function by each functional unit in this embodiment, please refer to the content described in the embodiment shown in fig. 1, which is not described herein again.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, in the embodiment of the present disclosure, the electronic device is a server in an embodiment of a result backtracking method, and the electronic device may be a tablet computer, a smart phone, a Personal Digital Assistant (PDA), or the like.
The electronic device may include: memory 102, process 101, communication interface 103, and a communication bus for enabling the connection communications of these components.
The Memory 102 is used to store all model data, the first corresponding relationship, and various data such as a computer program instruction corresponding to the result backtracking method and apparatus provided in the embodiment of the present application, where the Memory 102 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an erasable Read-Only Memory (EPROM), and the like.
The processor 101 is configured to execute the result backtracking method provided in the embodiment of the present application when reading and running the computer program instruction corresponding to the result backtracking method stored in the memory.
The processor 101 may be an integrated circuit chip having signal processing capability. The Processor 101 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also a Digital Signal Processor (DSP), discrete gate or transistor logic, discrete hardware components.
A communication interface 103, configured to receive the first trace-back request, and send the corresponding data source to the device that sent the first trace-back request.
In addition, a storage medium is provided in an embodiment of the present application, and a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer is caused to execute the method provided in any embodiment of the present application.
To sum up, under the condition that data sources are many and the calculation of the data sources is complex, the result backtracking of model data cannot be realized by using a traditional relational data base and using SQL statements, so that metadata corresponding to an identifier is quickly determined by using the corresponding relationship between the identifier based on data to be backtracked and the metadata of the data sources, and the data sources corresponding to the corresponding metadata are quickly found out from a data storage system by using a pre-trained big data platform.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based devices that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method for result backtracking, the method comprising:
receiving a first backtracking request sent by electronic equipment;
responding to the first backtracking request, and sending all pre-stored model data to the electronic equipment for display;
receiving a second backtracking request which is sent by the electronic equipment and represents that backtracking needs to be carried out on data to be backtracked in all the model data; the second backtracking request comprises an identifier of the data to be backtracked;
responding to the second backtracking request, and searching metadata corresponding to the identifier based on a first corresponding relation between the pre-stored identifier and metadata of the data source;
inputting the corresponding metadata into a big data platform trained in advance, so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata;
and sending the corresponding data source to the electronic equipment.
2. The method according to claim 1, wherein the second backtracking request further comprises: the data to be backtracked; before searching the metadata corresponding to the identifier, the method further comprises:
responding to the second backtracking request, and determining the data type of the algorithm matched with the data to be backtracked from the data types of the predetermined algorithms;
wherein sending the corresponding data source to the electronic device includes:
determining data matched with the data type of the matched algorithm from the corresponding data source based on the data type of the matched algorithm;
and sending the matched data to the electronic equipment.
3. The method of claim 1, wherein before receiving the first trace-back request sent by the electronic device, the method further comprises:
receiving a calculation request for representing that calculation needs to be carried out on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source;
in response to the calculation request, searching metadata corresponding to the name of the corresponding data source from a metadata base based on the name of the corresponding data source;
inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm;
storing the data to be backtracked into the data storage system;
distributing the same identification for the metadata corresponding to the name of the corresponding data source and the data to be backtracked;
storing the same identification with the first correspondence of metadata of the corresponding data source.
4. The method of claim 3, wherein after storing the data to be traced in the data storage system, the method further comprises:
generating metadata of the data to be backtracked;
storing the metadata of the data to be backtracked into the metadata database;
before all the pre-stored model data are sent to the electronic device, the method further comprises:
responding to the first backtracking request, and acquiring metadata of all model data from the metadata base;
and searching all model data corresponding to the metadata of all model data from the data storage system based on the metadata of all model data.
5. The method of claim 3, wherein prior to obtaining the computation request characterizing the computation required on the corresponding data source, the method further comprises:
acquiring the corresponding data source;
storing the corresponding data source into the data storage system to obtain metadata of the corresponding data source; wherein the metadata of the data source comprises the name of the corresponding data source;
and storing the metadata of the corresponding data source into the metadata database.
6. A result backtracking apparatus, the apparatus comprising:
the first receiving unit is used for receiving a first backtracking request sent by the electronic equipment;
the first response unit is used for responding to the first backtracking request and sending all pre-stored model data to the electronic equipment for display;
a second receiving unit, configured to receive a second trace-back request that represents that data to be traced back in all the model data needs to be traced back, where the second trace-back request is sent by the electronic device; the second backtracking request comprises an identifier of the data to be backtracked;
a second response unit, configured to respond to the second backtracking request, and search for metadata corresponding to the identifier based on a first correspondence between a pre-stored identifier and metadata of a data source;
the backtracking unit is used for inputting the corresponding metadata into a big data platform trained in advance so that the big data platform searches a data source corresponding to the corresponding metadata from a data storage system based on the corresponding metadata;
and the sending unit is used for sending the corresponding data source to the electronic equipment.
7. The apparatus of claim 6, further comprising:
a first determining unit, configured to determine, in response to the second backtracking request, a data type of an algorithm that matches the data to be backtracked from data types of predetermined algorithms;
the sending unit is specifically configured to determine, from the corresponding data source, data that matches the data type of the matched algorithm based on the data type of the matched algorithm; and sending the matched data to the electronic equipment.
8. The apparatus of claim 7, further comprising:
a third receiving unit, configured to receive a calculation request for representing that calculation needs to be performed on the corresponding data source; wherein, the calculation request comprises: the name and algorithm of the corresponding data source;
a third response unit, configured to, in response to the calculation request, find metadata corresponding to the name of the corresponding data source from a metadata database based on the name of the corresponding data source;
the output unit is used for inputting the corresponding metadata and the algorithm into the big data platform, so that the big data platform searches a data source corresponding to the corresponding metadata from the data storage system based on the corresponding metadata, and outputs the data to be backtracked based on the corresponding data source and the algorithm;
the first storage unit is used for storing the data to be backtracked into the data storage system;
the identification distribution unit is used for distributing the same identification for the metadata corresponding to the name of the corresponding data source and the data to be backtracked;
a second storage unit, configured to store the same identifier and the first corresponding relationship of the metadata of the corresponding data source.
9. An electronic device comprising a processor and a memory coupled to the processor, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method of any of claims 1-5.
10. A storage medium having stored thereon a computer program which, when run on a computer, causes the computer to perform the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387737.2A CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387737.2A CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111125472A true CN111125472A (en) | 2020-05-08 |
| CN111125472B CN111125472B (en) | 2023-05-02 |
Family
ID=70505999
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911387737.2A Active CN111125472B (en) | 2019-12-27 | 2019-12-27 | Result backtracking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111125472B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364096A (en) * | 2021-01-13 | 2021-02-12 | 天云融创数据科技（北京）有限公司 | High-dimensional metadata management system and method based on complex network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| US20180300086A1 (en) * | 2017-04-17 | 2018-10-18 | EMC IP Holding Company | Method and manager for managing storage system |
| CN110032575A (en) * | 2019-04-15 | 2019-07-19 | 网易（杭州）网络有限公司 | Data query method, apparatus, equipment and storage medium |
-
2019
- 2019-12-27 CN CN201911387737.2A patent/CN111125472B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| US20180300086A1 (en) * | 2017-04-17 | 2018-10-18 | EMC IP Holding Company | Method and manager for managing storage system |
| CN110032575A (en) * | 2019-04-15 | 2019-07-19 | 网易（杭州）网络有限公司 | Data query method, apparatus, equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364096A (en) * | 2021-01-13 | 2021-02-12 | 天云融创数据科技（北京）有限公司 | High-dimensional metadata management system and method based on complex network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111125472B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106296059B (en) | Method and equipment for determining delivery network points | |
| CN109766534B (en) | Report generation method and device, computer equipment and readable storage medium | |
| CN109344154B (en) | Data processing method, device, electronic equipment and storage medium | |
| WO2019080661A1 (en) | Method and device for identifying intention of user | |
| CN107633016A (en) | Data processing method and device and electronic equipment | |
| CN110555451A (en) | information identification method and device | |
| CN111159016A (en) | Standard detection method and device | |
| CN111445319A (en) | Voucher generation method and device, computer equipment and storage medium | |
| CN111858922A (en) | Service side information query method and device, electronic equipment and storage medium | |
| CN112231417A (en) | Data classification method and device, electronic equipment and storage medium | |
| CN113220657A (en) | Data processing method and device and computer equipment | |
| CN113704623A (en) | Data recommendation method, device, equipment and storage medium | |
| CN112182562A (en) | Data auditing method and system in sandbox environment, electronic equipment and medium | |
| CN112559526A (en) | Data table export method and device, computer equipment and storage medium | |
| CN111858834A (en) | Case dispute focus determining method, case dispute focus determining device, case dispute focus determining equipment and case dispute focus determining medium based on AI | |
| CN112330382A (en) | Item recommendation method and device, computing equipment and medium | |
| CN113434542B (en) | Data relationship identification method and device, electronic equipment and storage medium | |
| CN111125472A (en) | Result backtracking method and device, electronic equipment and storage medium | |
| CN114138243A (en) | Function calling method, device, equipment and storage medium based on development platform | |
| CN114003704A (en) | Method and device for creating designated tag guest group, electronic equipment and storage medium | |
| CN110020040B (en) | Method, device and system for querying data | |
| CN113705691A (en) | Image annotation checking method, device, equipment and medium based on artificial intelligence | |
| CN113434098A (en) | Printing function implementation method, device, electronic equipment, system and storage medium | |
| CN112347201A (en) | Information processing method, information processing device and terminal equipment | |
| CN112579781A (en) | Text classification method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |