CN112069174A - Data extraction method, device, equipment and storage medium - Google Patents
Data extraction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112069174A CN112069174A CN202010865370.7A CN202010865370A CN112069174A CN 112069174 A CN112069174 A CN 112069174A CN 202010865370 A CN202010865370 A CN 202010865370A CN 112069174 A CN112069174 A CN 112069174A
- Authority
- CN
- China
- Prior art keywords
- data
- target data
- target
- extraction
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Abstract
The embodiment of the invention discloses a data extraction method, a data extraction device, data extraction equipment and a storage medium. Wherein, the method comprises the following steps: responding to a data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule; generating at least two target data tables according to the target data structure of the target data source; and determining target data from the target data table according to a preset data extraction condition so as to realize the extraction of the target data. According to the embodiment of the invention, the incidence relation among a plurality of data tables is preset, so that the data information in various data sources and data structures can be rapidly and accurately mastered, the investment cost on data analysis is reduced, and the data extraction efficiency is improved.
Description
Technical Field
The embodiments of the present invention relate to big data technologies, and in particular, to a data extraction method, apparatus, device, and storage medium.
Background
With the development of physical interconnection, people have come to the era of data explosion, and the requirement for data extraction is higher and higher due to the increase of data volume.
In the prior art, when data of different sources or different structures are extracted, different data sources and data structures need to be distinguished and divided, the data are extracted respectively, and the data are analyzed and filtered after extraction, so that multi-source heterogeneous data cannot be extracted directly. Therefore, the operation process of data extraction is complicated, manpower and time are wasted, and the efficiency of data extraction is low.
Disclosure of Invention
The embodiment of the invention provides a data extraction method, a data extraction device, data extraction equipment and a storage medium, and aims to improve the efficiency of data extraction.
In a first aspect, an embodiment of the present invention provides a data extraction method, where the method includes:
responding to a data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule;
generating at least two target data tables according to the target data structure of the target data source;
and determining target data from the target data table according to a preset data extraction condition so as to realize the extraction of the target data.
In a second aspect, an embodiment of the present invention further provides a data extraction apparatus, where the apparatus includes:
the data source determining module is used for responding to the data extraction instruction and determining at least two target data sources with association relation based on a preset data association rule;
the data table generating module is used for generating at least two target data tables according to the target data structure of the target data source;
and the data extraction module is used for determining target data from the target data table according to preset data extraction conditions so as to extract the target data.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the data extraction method according to any embodiment of the present invention.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the data extraction method according to any of the embodiments of the present invention.
According to the embodiment of the invention, the multiple target data tables with the association relation are determined by acquiring the preset data association rule and the data extraction instruction, the data source of the target data tables can be heterogeneous data, and data filtering is performed on the multiple target data tables according to the preset data extraction condition to obtain the target data. The problem that data needs to be extracted from data of a single data source or a single data structure in the prior art is solved, dividing operation of the data source or the data structure is reduced, data extraction time is saved, and flexibility and efficiency of data extraction are improved.
Drawings
FIG. 1 is a schematic flow chart of a data extraction method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a data extraction method according to a second embodiment of the present invention;
fig. 3 is a block diagram of a data extraction apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a schematic flow chart of a data extraction method according to an embodiment of the present invention, which is applicable to data extraction and can be executed by a data extraction device. As shown in fig. 1, the method specifically includes the following steps:
and step 110, responding to the data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule.
The user sends a data extraction instruction to request for data extraction operation, and the request of the data extraction instruction may include a name of target data to be extracted. And determining the name of the target data to be extracted in response to a data extraction instruction of a user. And determining the target data according to the target data name. When the target data cannot be completely acquired from the data of one data source, data association rules may be preset, and when one of the target data sources is determined, another target data source may be acquired according to the data association rules, and the target data may be acquired from the acquired target data sources. For example, the data extraction instruction indicates that the name of target data to be acquired is the same-person data in the same train as a, one target data source obtained according to the name of the target data is the train number data table for a, and according to a preset data association rule, the data source associated with the train number data table can be determined to be a consumption data table of a certain part of residents in a city B, so that the two obtained target data sources are the train number data table and the consumption data table of the certain part of residents in the city B.
In this embodiment, optionally, before determining, in response to the data extraction instruction and based on a preset data association rule, that there are at least two target data sources with an association relationship, the method further includes: and in response to the determination instruction of the data association rule and the determination instruction of the data extraction condition, receiving an editing operation on the data association rule and the data extraction condition, and storing the data association rule and the data extraction condition.
Specifically, a user may preset a data association rule and a data extraction condition before extracting data, and associate two or more data sources, so that the user can automatically acquire an associated data source after obtaining one of the data sources. The data source may be an address source of the data, or may be a text, a picture, a database, or the like, in which the data is stored. And the data extraction conditions are used for filtering and screening the target data in the target data sources after the user acquires all the target data sources to obtain the target data meeting the conditions. The data extraction condition may indicate a target field to be extracted and may also indicate a data range of the target data, for example, the data extraction condition may be a consumption place of a target person at 10 to 11 am, the data extraction condition may include that the target field is the consumption place and the data range including the target data is the consumption place for consumption between 10 to 11 am. The user can edit the data association rule and the data extraction condition as required, after the data association rule and the data extraction condition are determined, the user can send a determination instruction of the data association rule and a determination instruction of the data extraction condition, the data association rule and the data extraction condition are stored, subsequent viewing and modification are facilitated, and the data association rule and the data extraction condition can be expressed by using an SQL (Structured Query Language) statement. The method has the advantages that the data association rule and the data extraction condition can be modified at any time, the modification operation is simple and rapid, the data extraction time is saved, and the flexibility and the efficiency of data extraction are improved.
In this embodiment, optionally, determining, in response to the data extraction instruction and based on a preset data association rule, at least two target data sources having an association relationship includes: determining a first target data source and a first target data source structure according to a target data name in the data extraction instruction; and determining at least one second target data source having an association relation with the first target data source and a second target data source structure of the second target data source according to the first target data source and a preset data association rule.
Specifically, a target data name in the data extraction instruction is obtained, a first target data source and a first target data source structure are determined according to the target data name, the first target data source can also be uploaded by a user when the user sends the data extraction instruction, and the data structure of the first target data source is the first target data source structure. According to the first target data source, one or more second target data sources matched with the first target data source are searched in a preset data association rule, and a second target data source structure is determined, wherein the data structure of the second target data source is the second target data source structure. The first target data source and the second target data source may be multi-source heterogeneous data, one data source may be associated with a plurality of other data sources, and the data source may be heterogeneous data, which means that different data sources may be structured data or unstructured data, and may be in the form of text or pictures, for example. The beneficial effects of setting up like this lie in, when associating to data, do not restrict the structural style of data, have realized combining and extracting multisource heterogeneous data, have solved the problem that can only extract to the same kind of structural data among the prior art, improve the flexibility and the efficiency of data extraction.
And 120, generating at least two target data tables according to the target data structure of the target data source.
The target data table stores data contents in target data sources, and one target data source may correspond to one target data table. And acquiring data contents in the target data sources, respectively storing the acquired data contents in the data tables, and generating the target data tables of the target data sources. The target data sources may be different target data structures, or different ways of generating the target data table.
In this embodiment, optionally, generating at least two target data tables according to the target data structure of the target data source includes: determining first target data and second target data according to the first target data source structure, the second target data source structure and a preset data reading algorithm; and determining a first target data table and a second target data table according to the first target data and the second target data.
Specifically, if the data of the target data source is structured data, the structured data can be directly placed in the data table; if the target data source structure is unstructured data, a preset data reading algorithm can be adopted to read the data content in the target data source, and then the data content is placed in the data table. For example, the target data source may be structured data of a database type, and the target data table may be directly obtained according to the content of the database; the target data source can be unstructured data in the form of pictures, and then the contents of characters, numbers and the like in the pictures can be read according to a data reading algorithm to generate a target data table. After determining the first target data source and the second target data source, the first target data source structure and the second target data source structure are validated. And determining a corresponding algorithm for extracting data content according to the confirmed target data source structure, and generating a first target data table and a second target data table. The beneficial effect that sets up like this lies in, realizes the processing to the heterogeneous data of multisource, makes the heterogeneous data of multisource convert the form of same data sheet into, reduces the operating procedure that data extraction, has solved the problem that can only carry out data extraction to same kind of structure data among the prior art, reduces the requirement to data structure, improves the efficiency and the flexibility of data extraction.
And step 130, determining target data from the target data table according to preset data extraction conditions to realize extraction of the target data.
The data extraction condition may indicate a specific requirement for the data to be extracted, for example, a target field and/or a data range of the target data may be determined according to the data extraction condition. And acquiring target data from the target data table according to the specific requirements on the data in the data extraction conditions.
In this embodiment, optionally, determining the target data from the target data table according to a preset data extraction condition includes: according to the data extraction conditions, filtering the data in the first target data table to obtain target fields and/or data ranges of the target data, which are associated with the data extraction conditions, in the first target data table; and filtering the data in the second target data table according to the target field and/or the data range and the data extraction condition to obtain target data.
Specifically, a data extraction condition is determined, and the data extraction condition may be that a user uploads when issuing a data extraction instruction. And acquiring a target data name in the data extraction condition, and determining a target field associated with the target data name from the first target data table. If the data extraction condition comprises the data range of the target data, determining whether the field of the data range of the target data is a field in the first target data table, and if so, acquiring data corresponding to the data range of the target data under the field in the first target data table; if not, the condition for the data range is temporarily ignored and the data range condition is matched to the second target data table. For example, if the data extraction condition indicates that the target data name is a consumption record of a co-traveler who takes the same train as a, the first target data table is a train number and passenger name data table, and the target fields associated with the data extraction condition are a train number field and a passenger name field, the co-traveler who takes the same train as a can be determined from the first target data table.
And matching the data extraction condition with the second target data table, determining the data content of a target field and/or a data range associated with the data extraction condition in the second target data table, and determining the intersection of the data content of the target field and/or the data range of the first target data table and the second target data table according to the data content of the target field and/or the data range of the first target data table to obtain target data. For example, the data extraction condition indicates that the target data name is consumption records between 10 and 11 of the same passenger as a on the same train, namely the data extraction condition is to search the consumption records between 10 and 11 of the same passenger as a on the same train. And obtaining a list of the same pedestrians who take the same train as the train A according to the first target data table. And recording consumption records of different people in the second target data table, acquiring consumption records of which the data range is between 10 and 11 points according to the data extraction conditions, and acquiring consumption records of which the data range is between 10 and 11 points of the same person who takes the same train as A in the second target data table according to the data extracted from the first target data table. The beneficial effect who sets up like this lies in, can utilize multisource data to carry out data extraction, avoids data to miss, improves the accuracy nature of data extraction.
According to the technical scheme of the embodiment, a plurality of target data tables with association relations are determined by obtaining preset data association rules and data extraction instructions, the data sources of the target data tables can be heterogeneous data, and data filtering is performed on the plurality of data tables according to preset data extraction conditions to obtain the target data. The problem that data needs to be extracted from data of a single data source or a single data structure in the prior art is solved, dividing operation of the data source or the data structure is reduced, data extraction time is saved, and flexibility and efficiency of data extraction are improved.
Example two
Fig. 2 is a schematic flow chart of a data extraction method according to a second embodiment of the present invention, and the present embodiment is further optimized based on the above embodiments. As shown in fig. 2, the method specifically includes the following steps:
After the target data table corresponding to the target data source is obtained, all field data in each target data table are obtained, and all field data are stored in one temporary data table, that is, the contents of a plurality of data tables can be combined into one data table. The method and the device avoid extraction from multiple data tables when the target data is extracted, prevent data omission and improve the accuracy and efficiency of data extraction.
And 240, determining target data from the temporary data table according to a preset data extraction condition so as to realize the extraction of the target data.
And after the temporary data table is obtained, acquiring target data meeting the data extraction conditions from the temporary data table according to preset data extraction conditions. If the temporary data table is not generated, the data needs to be extracted from the first target data table and the second target data table respectively.
In this embodiment, optionally, determining the target data from the target data table according to a preset data extraction condition includes: determining a target field of the target data and/or a data range of the target data according to the data extraction condition; and filtering the data in the temporary data table according to the target field and/or the data range to obtain target data.
Specifically, the specific requirements for extracting data are determined according to preset data extraction conditions, and the target field of the target data and/or the data range of the target data are obtained according to the incidence relation between each field in the temporary data table and the specific requirements for extracting data. For example, if the data extraction condition is to extract consumption records between 10 and 11 points of the same passenger who takes the same train as a, the obtained target fields may include the number of cars, the name of the passenger, the consumption record, the consumption time, and the like according to the temporary data table, and the data range of the consumption record is between 10 and 11 points. And screening or filtering the temporary data table according to the acquired target field and/or data range, and acquiring data meeting the target field and/or data range from the temporary data table to finish the extraction of the target data. For example, data intersections may be found from the determined target fields and/or data ranges. The beneficial effect who sets up like this lies in, can carry out data extraction through a temporary data table, avoids carrying out the operation of drawing many times between many data tables, practices thrift data extraction time, improves data extraction efficiency.
According to the embodiment of the invention, a plurality of target data tables with association relation are determined by acquiring the preset data association rule and the data extraction instruction, and the data source of the target data tables can be heterogeneous data. And forming a temporary data table by the plurality of target data tables, and filtering data from the temporary data table according to a preset data extraction condition to obtain target data. The problem of among the prior art, need extract from the data of single data source or single data structure is solved, reduce the division operation of data source or data structure, avoid carrying out iterative extraction operation to many data tables, practice thrift the data and draw time, improved the flexibility and the efficiency of data extraction.
EXAMPLE III
Fig. 3 is a block diagram of a data extraction device according to a third embodiment of the present invention, which is capable of executing a data extraction method according to any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 3, the apparatus specifically includes:
the data source determining module 301 is configured to determine, in response to a data extraction instruction, at least two target data sources having an association relationship based on a preset data association rule;
a data table generating module 302, configured to generate at least two target data tables according to a target data structure of the target data source;
the data extraction module 303 is configured to determine target data from the target data table according to a preset data extraction condition, so as to extract the target data.
Optionally, the apparatus further comprises:
and the rule and condition storage module is used for responding to a determination instruction of the data association rule and a determination instruction of the data extraction condition, receiving editing operation of the data association rule and the data extraction condition and storing the data association rule and the data extraction condition before responding to a data extraction instruction and determining at least two target data tables with association relation based on a preset data association rule.
Optionally, the data source determining module 301 includes:
the first data source determining unit is used for determining a first target data source and a first target data source structure according to the target data name in the data extracting instruction;
and the second data source determining unit is used for determining at least one second target data source which has an association relation with the first target data source and a second target data source structure of the second target data source according to the first target data source and a preset data association rule.
Optionally, the data table generating module 302 includes:
the data determining unit is used for determining first target data and second target data according to the first target data source structure, the second target data source structure and a preset data reading algorithm;
and the data table determining unit is used for determining the first target data table and the second target data table according to the first target data and the second target data.
Optionally, the apparatus further comprises:
the field data acquisition module is used for acquiring field data in at least two target data tables after generating the at least two target data tables according to the target data structure of the target data source;
the temporary table generating module is used for generating a temporary data table according to the field data; wherein, the temporary data table includes the data of each field.
Optionally, the data extracting module 303 is specifically configured to:
according to the data extraction conditions, filtering the data in the first target data table to obtain target fields and/or data ranges of the target data, which are associated with the data extraction conditions, in the first target data table;
and filtering the data in the second target data table according to the target field and/or the data range and the data extraction condition to obtain target data.
Optionally, the data extracting module 303 is further specifically configured to:
determining a target field of the target data and/or a data range of the target data according to the data extraction condition;
and filtering the data in the temporary data table according to the target field and/or the data range to obtain target data.
According to the embodiment of the invention, the multiple target data tables with the association relation are determined by acquiring the preset data association rule and the data extraction instruction, the data source of the target data tables can be heterogeneous data, and data filtering is performed on the multiple target data tables according to the preset data extraction condition to obtain the target data. The problem that data needs to be extracted from data of a single data source or a single data structure in the prior art is solved, dividing operation of the data source or the data structure is reduced, data extraction time is saved, and flexibility and efficiency of data extraction are improved.
Example four
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary computer device 400 suitable for use in implementing embodiments of the present invention. The computer device 400 shown in fig. 4 is only an example and should not bring any limitations to the functionality or scope of use of the embodiments of the present invention.
As shown in fig. 4, computer device 400 is in the form of a general purpose computing device. The components of computer device 400 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 that couples the various system components (including the system memory 402 and the processing unit 401).
The system memory 402 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)404 and/or cache memory 405. The computer device 400 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 403 by one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 407 generally perform the functions and/or methods of the described embodiments of the invention.
The computer device 400 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), with one or more devices that enable a user to interact with the computer device 400, and/or with any devices (e.g., network card, modem, etc.) that enable the computer device 400 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interface 411. Moreover, computer device 400 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with the other modules of computer device 400 over bus 403. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 401 executes various functional applications and data processing by running a program stored in the system memory 402, for example, to implement a data extraction method provided by an embodiment of the present invention, including:
responding to a data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule;
generating at least two target data tables according to a target data structure of a target data source;
and determining target data from the target data table according to preset data extraction conditions so as to realize extraction of the target data.
EXAMPLE five
The fifth embodiment of the present invention further provides a storage medium containing computer-executable instructions, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the data extraction method provided in the fifth embodiment of the present invention is implemented, where the computer program includes:
responding to a data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule;
generating at least two target data tables according to a target data structure of a target data source;
and determining target data from the target data table according to preset data extraction conditions so as to realize extraction of the target data.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method of data extraction, comprising:
responding to a data extraction instruction, and determining at least two target data sources with association relation based on a preset data association rule;
generating at least two target data tables according to the target data structure of the target data source;
and determining target data from the target data table according to a preset data extraction condition so as to realize the extraction of the target data.
2. The method of claim 1, before determining that there are at least two target data sources of an association relationship based on a preset data association rule in response to a data extraction instruction, further comprising:
and in response to the determination instruction of the data association rule and the determination instruction of the data extraction condition, receiving an editing operation on the data association rule and the data extraction condition, and storing the data association rule and the data extraction condition.
3. The method of claim 1, wherein determining at least two target data sources with association based on a preset data association rule in response to a data extraction instruction comprises:
determining a first target data source and a first target data source structure according to a target data name in the data extraction instruction;
and determining at least one second target data source having an association relation with the first target data source and a second target data source structure of the second target data source according to the first target data source and a preset data association rule.
4. The method of claim 3, wherein generating at least two target data tables based on the target data structure of the target data source comprises:
determining first target data and second target data according to the first target data source structure, the second target data source structure and a preset data reading algorithm;
and determining a first target data table and a second target data table according to the first target data and the second target data.
5. The method of claim 1, after generating at least two target data tables based on the target data structure of the target data source, further comprising:
acquiring data of each field in the at least two target data tables;
generating a temporary data table according to the field data; wherein, the temporary data table comprises the data of each field.
6. The method of claim 4, wherein determining target data from the target data table according to preset data extraction conditions comprises:
according to the data extraction condition, filtering data in a first target data table to obtain a target field associated with the data extraction condition and/or a data range of the target data in the first target data table;
and filtering the data in the second target data table according to the target field and/or the data range and the data extraction condition to obtain the target data.
7. The method of claim 5, wherein determining target data from the target data table according to preset data extraction conditions comprises:
determining a target field of the target data and/or a data range of the target data according to the data extraction condition;
and filtering the data in the temporary data table according to the target field and/or the data range to obtain the target data.
8. A data extraction apparatus, comprising:
the data source determining module is used for responding to the data extraction instruction and determining at least two target data sources with association relation based on a preset data association rule;
the data table generating module is used for generating at least two target data tables according to the target data structure of the target data source;
and the data extraction module is used for determining target data from the target data table according to preset data extraction conditions so as to extract the target data.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the data extraction method according to any one of claims 1-7 when executing the program.
10. A storage medium containing computer-executable instructions for performing the data extraction method of any one of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010865370.7A CN112069174A (en) | 2020-08-25 | 2020-08-25 | Data extraction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010865370.7A CN112069174A (en) | 2020-08-25 | 2020-08-25 | Data extraction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112069174A true CN112069174A (en) | 2020-12-11 |
Family
ID=73659340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010865370.7A Pending CN112069174A (en) | 2020-08-25 | 2020-08-25 | Data extraction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069174A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783543A (en) * | 2019-01-14 | 2019-05-21 | 广州虎牙信息科技有限公司 | Data query method, apparatus, equipment and storage medium |
| CN110413634A (en) * | 2019-06-27 | 2019-11-05 | 北京奇艺世纪科技有限公司 | Data query method, system, device and computer readable storage medium |
| CN111104426A (en) * | 2019-11-22 | 2020-05-05 | 深圳智链物联科技有限公司 | Data query method and system |
| CN111414352A (en) * | 2020-03-27 | 2020-07-14 | 北京明略软件系统有限公司 | Database information management method and device |
-
2020
- 2020-08-25 CN CN202010865370.7A patent/CN112069174A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783543A (en) * | 2019-01-14 | 2019-05-21 | 广州虎牙信息科技有限公司 | Data query method, apparatus, equipment and storage medium |
| CN110413634A (en) * | 2019-06-27 | 2019-11-05 | 北京奇艺世纪科技有限公司 | Data query method, system, device and computer readable storage medium |
| CN111104426A (en) * | 2019-11-22 | 2020-05-05 | 深圳智链物联科技有限公司 | Data query method and system |
| CN111414352A (en) * | 2020-03-27 | 2020-07-14 | 北京明略软件系统有限公司 | Database information management method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180314882A1 (en) | Sorting and displaying digital notes on a digital whiteboard | |
| WO2018072071A1 (en) | Knowledge map building system and method | |
| US11244102B2 (en) | Systems and methods for facilitating data object extraction from unstructured documents | |
| JP6756079B2 (en) | Artificial intelligence-based ternary check method, equipment and computer program | |
| CN109522312A (en) | A kind of data processing method, device, server and storage medium | |
| US20140122455A1 (en) | Systems and Methods for Intelligent Parallel Searching | |
| CN107909088B (en) | Method, apparatus, device and computer storage medium for obtaining training samples | |
| WO2019100619A1 (en) | Electronic apparatus, method and system for multi-table correlated query, and storage medium | |
| CN111597800B (en) | Method, device, equipment and storage medium for obtaining synonyms | |
| CN111506603B (en) | Data processing method, device, equipment and storage medium | |
| CN111694866A (en) | Data searching and storing method, data searching system, data searching device, data searching equipment and data searching medium | |
| US10552394B2 (en) | Data storage with improved efficiency | |
| CN110990406B (en) | Fuzzy query method, device, equipment and medium | |
| CN113962597A (en) | Data analysis method and device, electronic equipment and storage medium | |
| CN111198917A (en) | Data processing method, device, equipment and storage medium | |
| CN115759040A (en) | Electronic medical record analysis method, device, equipment and storage medium | |
| CN112069174A (en) | Data extraction method, device, equipment and storage medium | |
| CN115114297A (en) | Data lightweight storage and search method and device, electronic equipment and storage medium | |
| CN114154468A (en) | Report synthesis method and device, electronic equipment and storage medium | |
| CN113127496B (en) | Method and device for determining change data in database, medium and equipment | |
| CN111639173B (en) | Epidemic situation data processing method, device, equipment and storage medium | |
| CN113886425A (en) | Data processing method, device, equipment and storage medium | |
| CN114020774A (en) | Method, device and equipment for processing multiple rounds of question-answering sentences and storage medium | |
| CN109460511B (en) | Method and device for acquiring user portrait, electronic equipment and storage medium | |
| CN114118072A (en) | Document structuring method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |