CN110502513A - Collecting method, device, equipment and computer readable storage medium - Google Patents

Collecting method, device, equipment and computer readable storage medium Download PDF

Info

Publication number
CN110502513A
CN110502513A CN201910761596.XA CN201910761596A CN110502513A CN 110502513 A CN110502513 A CN 110502513A CN 201910761596 A CN201910761596 A CN 201910761596A CN 110502513 A CN110502513 A CN 110502513A
Authority
CN
China
Prior art keywords
data
tables
demand
acquisition
code
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910761596.XA
Other languages
Chinese (zh)
Inventor
刘雪花
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Property and Casualty Insurance Company of China Ltd
Original Assignee
Ping An Property and Casualty Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Property and Casualty Insurance Company of China Ltd filed Critical Ping An Property and Casualty Insurance Company of China Ltd
Priority to CN201910761596.XA priority Critical patent/CN110502513A/en
Publication of CN110502513A publication Critical patent/CN110502513A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24553Query execution of query operations
    • G06F16/24558Binary matching operations
    • G06F16/2456Join operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/252Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application

Abstract

The present invention discloses a kind of collecting method, device, equipment and computer readable storage medium, the described method includes: when receiving demand acquisition document, read the demand field information in the demand acquisition document, and call default foundation code, the demand field information is added in the connection sentence of the default foundation code, generates acquisition code;Acquisition code operation is controlled, generates operation result, and aiming field corresponding with the demand field information is determined according to the operation result, and tables of data corresponding with each aiming field;According to the quantity of the tables of data, the tables of data is formed into target matrix, and screen to the target matrix, acquire data corresponding with the demand field information.The efficiency of data acquisition can be improved according to its progress data acquisition based on the tables of data that the target matrix that big data processing technique generates is the corresponding place of demand field information for needing to acquire in this programme.

Description

Collecting method, device, equipment and computer readable storage medium
Technical field
The invention mainly relates to technical field of data processing, specifically, being related to a kind of collecting method, device, setting Standby and computer readable storage medium.
Background technique
Currently, each data in database are stored according to classification of type, the number of same type for the standardized administration of data According to being stored in same tables of data, and different data are stored in different tables of data.When being acquired to data, adopted Collect the type either single of data or a variety of, be first associated each tables of data in database, and then from associated Searching in each tables of data needs each data acquired to be acquired.
But it in such a way that associated each tables of data carries out data search and acquires, is deposited in associated each tables of data In the tables of data unrelated with required acquisition data;Simultaneously because of the association to extraneous data table, so that carrying out search operation When, it is also necessary to this, unrelated tables of data is searched;Association and search operation to extraneous data table, consume resource, drop The efficiency of low data acquisition.
Summary of the invention
The main object of the present invention is to provide a kind of collecting method, device, equipment and computer readable storage medium, The prior art is aimed to solve the problem that in data acquisition, association and search operation to extraneous data table consume resource, reduce number The problem of according to collecting efficiency.
To achieve the above object, the present invention provides a kind of collecting method, and the collecting method includes following step It is rapid:
When receiving demand acquisition document, the demand field information in the demand acquisition document is read, and call pre- If foundation code, the demand field information is added in the connection sentence of the default foundation code, generates acquisition code;
The acquisition code operation is controlled, generates operation result, and according to operation result determination and the demand word The corresponding aiming field of segment information, and tables of data corresponding with each aiming field;
According to the quantity of the tables of data, the tables of data is formed into target matrix, and to the target matrix into Row screening, acquires data corresponding with the demand field information.
Preferably, described that aiming field corresponding with the demand field information is determined according to the operation result, and The step of tables of data corresponding with each aiming field includes:
Read pair of full word section comparison in demand field information described in the operation result and the default foundation code Than as a result, and according to the comparing result, determine the corresponding aiming field of demand field information described in the full word Duan Zhongyu;
The corresponding data table name of each aiming field in the operation result is read, and according to the data table name, really Fixed tables of data corresponding with each aiming field.
Preferably, the quantity according to the tables of data, by the tables of data formed target matrix the step of include:
Judge whether the corresponding data table name of each aiming field has uniqueness, if having uniqueness, determines institute State tables of data quantity be it is single, otherwise determine the tables of data quantity be it is multiple;
When the quantity of the tables of data is single, then the single tables of data is determined as target matrix;
When the quantity of the tables of data is multiple, then reads and store Various types of data correlation in multiple tables of data Correlated identities, and the correlated identities are generated as association identification, multiple tables of data are associated with, target matrix is formed.
Preferably, described that the target matrix is screened, acquire data corresponding with the demand field information The step of include:
The screening range information in the demand acquisition document is read, and according to the screening range information to the target Tables of data is screened for the first time, generates first garbled data;
According to tables of data corresponding with each aiming field, each first garbled data is screened again, it is raw At garbled data again, and the garbled data again is set as data corresponding with the demand field information and is acquired.
Preferably, described the garbled data again is set as data corresponding with the demand field information to be acquired The step of include:
Judge whether the target matrix carries the association identification, if the association identification is carried, according to The garbled data again is associated with by association identification, and by garbled data is set as and the demand word again described in after association The corresponding data of segment information are acquired;
If not carrying the association identification, the garbled data again is set as corresponding with the demand field information Data are acquired.
Preferably, described that the demand field information is added in the connection sentence in the default foundation code, it is raw Include: at the step of acquisition code
According to default connection identifier, the connection sentence in the default foundation code is determined, and identified according to design variables, Detect the variable position in the connection sentence;
The demand field information is added in the variable position, replaces the variable in the variable position, and examine Survey whether the demand field information is added in the variable position;
If being added in the variable position, the default foundation code is updated to acquisition code.
Preferably, include: before the step of control acquisition code operation, generation operation result
The acquisition code and presets rule are compared, judge whether each sentence in the acquisition code meets institute State presets rule, wherein the presets rule is used to judge the correctness of the acquisition code;
If each sentence meets the presets rule, the control acquisition code operation is executed, fortune is generated The step of row result;
If there is the abnormal statement for not meeting the presets rule in each sentence, it is based on the abnormal statement Output prompt modification information.
In addition, to achieve the above object, the present invention also proposes a kind of data acquisition device, the data acquisition device packet It includes:
Read module, for reading the demand field in the demand acquisition document when receiving demand acquisition document Information, and default foundation code is called, the demand field information is added in the connection sentence of the default foundation code, Generate acquisition code;
Control module generates operation result, and determine according to the operation result for controlling the acquisition code operation Aiming field corresponding with the demand field information, and tables of data corresponding with each aiming field;
The tables of data is formed target matrix, and to described for the quantity according to the tables of data by acquisition module Target matrix is screened, and data corresponding with the demand field information are acquired.
In addition, to achieve the above object, the present invention also proposes a kind of data acquisition equipment, the data acquisition equipment packet It includes: memory, processor, communication bus and the data acquisition program being stored on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the data acquisition program, to perform the steps of
When receiving demand acquisition document, the demand field information in the demand acquisition document is read, and call pre- If foundation code, the demand field information is added in the connection sentence of the default foundation code, generates acquisition code;
The acquisition code operation is controlled, generates operation result, and according to operation result determination and the demand word The corresponding aiming field of segment information, and tables of data corresponding with each aiming field;
According to the quantity of the tables of data, the tables of data is formed into target matrix, and to the target matrix into Row screening, acquires data corresponding with the demand field information.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium Storage medium be stored with one perhaps more than one program the one or more programs can by one or one with On processor execute to be used for:
When receiving demand acquisition document, the demand field information in the demand acquisition document is read, and call pre- If foundation code, the demand field information is added in the connection sentence of the default foundation code, generates acquisition code;
The acquisition code operation is controlled, generates operation result, and according to operation result determination and the demand word The corresponding aiming field of segment information, and tables of data corresponding with each aiming field;
According to the quantity of the tables of data, the tables of data is formed into target matrix, and to the target matrix into Row screening, acquires data corresponding with the demand field information.
The collecting method of the present embodiment, when receiving demand acquisition document, first to demand field information therein It is read out, recalls default foundation code, and demand field information is added in the connection sentence of default foundation code, it is raw At acquisition code;Hereafter control acquisition code operation generates operation result, and according to operation result determination and demand field information Corresponding aiming field, and tables of data corresponding with each aiming field;And then according to the quantity of tables of data, tables of data is formed Target matrix, and target matrix is screened, it realizes and data corresponding with demand field information is acquired.This programme will The data for needing to acquire form demand acquisition document, and demand field information therein characterizes the required data class acquired Type;Demand field information is added to acquisition code formed in default foundation code to be used to look into demand field information It looks for, to determine the tables of data where demand field information;And then screen tables of data formation target matrix, it acquires and needs Seek the corresponding data of field information.Because target matrix is formed as the tables of data where the demand field information for needing to acquire, keep away Exempt to save resource to the association and search operation of the tables of data unrelated with acquisition data, has improved the efficiency of data acquisition; Acquisition code is formed by by default foundation code simultaneously, lookup only is compared to demand field information at runtime, therefore is right The data volume of ratio is small, and search speed is fast.
Detailed description of the invention
Fig. 1 is the flow diagram of collecting method first embodiment of the invention;
Fig. 2 is the functional block diagram of data acquisition device first embodiment of the invention;
Fig. 3 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The present invention provides a kind of collecting method.
Fig. 1 is please referred to, Fig. 1 is the flow diagram of collecting method first embodiment of the present invention.In the present embodiment, The collecting method includes:
Step S10 reads the demand field information in the demand acquisition document when receiving demand acquisition document, And default foundation code is called, the demand field information is added in the connection sentence of the default foundation code, is generated Acquire code;
Collecting method of the invention is applied to server, is suitable for carrying out various types of data by server Acquisition.For the ease of the acquisition of Various types of data, it is provided with document collection mechanism;Collector's setting with data acquisition demand Demand acquires document, by the demand field information of acquisition data type required for characterization, such as number of policy, warrantee's field;With And characterization needs to acquire the screening range information of data location, such as time range, facility-wide, inventory coding range, industry Business person's coding range etc. is arranged into blank document, and formation demand acquisition document is simultaneously uploaded onto the server.When server receives When the demand acquires document, first demand field information therein is read out, with true by the demand field information of the reading The data type acquired required for fixed.Default foundation code is also preset simultaneously, which includes database The table name of tables of data where middle stored all fields and each field;Each field having in characterization database, with And data table name corresponding to each field.In addition, being additionally provided with connection sentence in default foundation code, demand acquisition is being received Document when needing to carry out data acquisition, is called default foundation code, and the demand field information of reading is added to pre- If in the connection sentence of foundation code.Demand field information is added to the default foundation code after connecting in sentence by this, The code that substantially tables of data where demand field information is searched, as acquisition code, to believe demand field The data that need to acquire of breath characterization are acquired.In view of the code statement for presetting involved in foundation code is numerous, need First from connection sentence is wherein told, resolution operation can be carried out by connection identifier;Specifically, demand field information is added Into the connection sentence in default foundation code, generating the step of acquiring code includes:
Step S11 determines the connection sentence in the default foundation code according to default connection identifier, and according to default Marking variable detects the variable position in the connection sentence;
The connection identifier of characterization connection sentence is preset, and in default foundation code development process, to company therein It connects sentence and adds the connection identifier.During being added to demand field information, first detect pre- in default foundation code If the code statement where connection identifier, the code statement where this is to connect sentence.Furthermore it includes more for connecting in sentence A code, and the conventional code of its function is realized, and be used for replacement demand comprising supporting connection sentence operation in multiple codes Field information, with the variant code inquired demand field information.Conventional code has not replaceable characteristic, and variable generation Code has replaceable characteristic, and variant code is identified with pre-set design variables and characterized.In determining default foundation code Connection sentence and then to wherein with design variables mark code detect, this have design variables identify generation Code is variant code;It is determined as the position where variant code in connection sentence to connect the variable position in sentence, with right Demand field information is inquired.
The demand field information is added in the variable position, replaces the change in the variable position by step S12 Amount, and detect whether the demand field information is added in the variable position;
The default foundation code is updated to acquisition code if being added in the variable position by step S13.
Further, the variable that possessed variant code is replaced required for being in variable position, in the company of determination After connecing the variable position in sentence, each field in demand field information is added in the variable position, change therein is replaced Amount.Because field involved by demand field information is numerous, the blank character for distinguishing each demand field information can be preset, As "/", ";" etc.;In addition, first by variable deletion possessed in variable position, then each demand field information added one by one It is added in the variable position, and be split with blank character, to ensure the independence in demand field information between each field.This In external adding procedure, detect whether each field in demand field information is added in variable position, to prevent from omitting;The inspection Survey can be carried out by the field quantity and field repeatability of demand field information, that is, judge the word of demand field information added Segment number with from demand acquisition document read field quantity it is whether consistent, and demand field information added by judging In whether there is duplicate demand field information.Believe when duplicate demand field is not present in added demand field information Breath, and field quantity is consistent, then illustrates that each field in demand field information is added in variable position;Otherwise it weighs if it exists The demand field information or field quantity added again is inconsistent, then illustrates there is the demand being not yet added in variable position Field information needs to continue to add, until each field in demand field information is added in variable position.And it is detecting After each field in demand field information is added in variable position, then by the default basic generation added with demand field information Code is updated to acquisition code, to inquire tables of data corresponding to wherein demand field information, and then adopts from tables of data Collection demand field information is to the data in requisition for acquisition.
Step S20 controls the acquisition code operation, generates operation result, and according to the operation result determination and institute State the corresponding aiming field of demand field information, and tables of data corresponding with each aiming field;
Further, after generating the acquisition code added with demand field information, server is sent to acquisition code Operating instruction, control acquisition code operation;Code is acquired according to itself pre-set logic operation, to demand field information Corresponding field is searched, and operation result is generated.Server is read out the operation result, and ties according to lookup therein Fruit determines corresponding with demand field information aiming field, and with tables of data corresponding to each aiming field.Default basis generation Code in include all fields stored in database, acquire code operate to by demand field information each field and All fields compare, from the process searched in all fields with each field corresponding field in demand field information;Operation Operation result generated includes the comparing result of the comparison, and server can determine according to the comparing result to be believed with demand field The corresponding each aiming field of breath;Specifically, aiming field corresponding with demand field information is determined according to operation result, and The step of tables of data corresponding with each aiming field includes:
Step S21 reads full word section in demand field information described in the operation result and the default foundation code The comparing result of comparison, and according to the comparing result, determine the corresponding mesh of demand field information described in the full word Duan Zhongyu Marking-up section;
Further, using all fields stored in the database that default foundation code includes as default foundation code In full word section, demand field information and the full word section are compared by acquisition code operation, it is each to search full word Duan Zhongyu The corresponding field of demand field information;Hereafter mark is searched to each corresponding field distribution that lookup obtains, as aiming field The comparing result being generated as in operation result.Server is read out comparing result therein after reading operation result, Aiming field corresponding with demand field information is determined by the lookup mark in comparing result.
Step S22 reads the corresponding data table name of each aiming field in the operation result, and according to the data Table name, determining tables of data corresponding with each aiming field.
Further, because other than including all fields stored in database, being gone back in default foundation code Include the table name of tables of data where each field, and has between each field and its respectively table name of place tables of data corresponding Relationship;As the field for being included has a W1 and W2, and the table name of tables of data where W1 is known as M1, and the table name of tables of data is known as where W2 M2 then has corresponding relationship between W1 and M1 and between W2 and M2.It acquires code to run in operation result generated, remove It also include corresponding to each aiming field except comparing result comprising characterizing aiming field corresponding to each demand field information Table name, to characterize the tables of data where each aiming field.By table name read corresponding with each aiming field in operation result It is taken as data table name, and determines tables of data corresponding with each aiming field according to the data table name.Because different data table is had Table name it is different, the tables of data that data table name is characterized, tables of data as corresponding with each aiming field characterizes each target word Tables of data where section.
The tables of data is formed target matrix, and to the target according to the quantity of the tables of data by step S30 Tables of data is screened, and data corresponding with the demand field information are acquired.
Understandably, because the data type of acquisition required for demand field information characterization is numerous, so that with demand field The corresponding aiming field of information is numerous, and multiple aiming field is likely located at the same tables of data, it is also possible to be located at multiple and different Tables of data;Tables of data i.e. corresponding with aiming field may be single, it is also possible to be multiple.For the ease of the acquisition of data, The tables of data of the single or multiple quantity is formed into target matrix, and target matrix is screened, acquisition and demand word The corresponding data of segment information, i.e. demand acquire the data of required acquisition in document.Because tables of data may be that individually may also be It is multiple, when forming target matrix, the otherness according to quantity is needed to carry out different operations;Specifically, according to tables of data Quantity, by tables of data formed target matrix the step of include:
Step S31, judges whether the corresponding data table name of each aiming field has uniqueness, if having uniqueness, Then determine the tables of data quantity be it is single, otherwise determine the tables of data quantity be it is multiple;
Further, because table name possessed by between different data table is different, the uniqueness of data table name can be first passed through To determine that the quantity of tables of data is single or multiple;Whether data table name corresponding to each aiming field is uniquely sentenced It is disconnected, if corresponding data table name only one, that is, there is uniqueness, then illustrate that the quantity of tables of data is single;And if corresponding Data table name have it is multiple, that is, do not have uniqueness, then illustrate tables of data quantity be it is multiple.
The single tables of data is then determined as target data when the quantity of the tables of data is single by step S32 Table;
Further, when being judged that there is data table name uniqueness to illustrate that demand is adopted when the quantity of tables of data is single The data acquired required for set document are in the individual data table;I.e. by the individual data table can collect institute it is in need The data of acquisition, to include individual tables of data of all data of required acquisition as target matrix using this.
Step S33 then reads in multiple tables of data when the quantity of the tables of data is multiple and stores Various types of data The correlated identities of correlation, and the correlated identities are generated as association identification, multiple tables of data are associated with, target is formed Tables of data.
And if judge that data table name does not have uniqueness, when tables of data corresponding to aiming field is multiple, then illustrate The data acquired required for demand acquisition document are present in multiple different tables of data, need to adopt respectively from multiple tables of data Various data required for collecting.The correlated identities between multiple tables of data are read at this time, which characterizes each tables of data Correlation between middle stored Various types of data.If corresponding tables of data includes insurer's information table and insured amount table, People from China National Investment & Guaranty Corp.'s information table includes the declaration form number that insurer is insured, and insured amount is the gold insured for each declaration form Volume has correlation with the declaration form number of each declaration form;To declaration form number be insurer's information table and insured amount table it Between correlated identities.The correlated identities of reading are generated as association identification, and multiple tables of data are closed with the association identification Connection generates target matrix;The association identification is distributed to each corresponding tables of data, to establish association between multiple tables of data Relationship forms the target matrix where all data that characterization demand acquisition document needs to acquire.In demand acquisition document Demand field information be a1, a2 and a3, if through determine a1, a2, a3 where tables of data be respectively A, B, C, to tables of data A, B, C distribute association identification, three are associated with to form target matrix, to acquire required each Field Count from three According to;And if the tables of data where being determined a1, a2, a3 is D, and D is directly set as target matrix, with from wherein acquiring institute Each field data of demand.
Understandably, data involved in target matrix are numerous, acquire including demand and adopt required for document The Various types of data of collection also includes other kinds of data;If the data of the demand acquisition required acquisition of document are January 1 in 2018 The data stored after day, and may include the data before this date in target matrix.To meet to collect The data of demand need to need to acquire according to characterization in demand acquisition document the screening range information of data location, to mesh Mark tables of data is screened;Specifically, target matrix is screened, acquires the step of data corresponding with demand field information Suddenly include:
Step S34 reads the screening range information in the demand acquisition document, and according to the screening range information pair The target matrix is screened for the first time, generates first garbled data;
Further, first the screening range information in demand acquisition document is read out, to pass through the screening of the reading Range information determines the corresponding range of data for needing to acquire;Again according to the screening range information of the reading to target matrix into The first screening of row filters out from target matrix and meets the corresponding desired first garbled data of screening range information.Wherein just The secondary obtained first garbled data of screening is different according to the difference for screening range information, such as when screening range information is the time Range, the then first garbled data filtered out are all data generated within this time range in target matrix;And When screening range information is business personnel's coding, then the first garbled data filtered out is to compile in target matrix with the business personnel All data corresponding to code.
Step S35 carries out again each first garbled data according to tables of data corresponding with each aiming field Screening, generates garbled data again, and by the garbled data again be set as data corresponding with the demand field information into Row acquisition.
Further, because screening the institute in obtained first garbled data to meet screening range information requirement for the first time There are data, include data corresponding with each aiming field in all data, also includes data not corresponding with each aiming field; To needing to screen first garbled data again according to each aiming field, with from wherein filtering out and each aiming field pair The data answered.It is being filtered out again with each mesh because having corresponding relationship between each aiming field and demand field information The corresponding data of marking-up section are as adopted required for demand acquisition document substantially with data corresponding to demand field information The data of collection.Because tables of data corresponding to each aiming field may be single, it is also possible to it is multiple, and the single or multiple tables of data shapes As target matrix, first garbled data obtained by first screening is carried out to target matrix and is similarly probably derived from individually Tables of data rice industry is probably derived from multiple tables of data.When the corresponding tables of data of each aiming field is single, then the individual data Table directly forms target matrix, and first garbled data then derives from the individual data table;When the corresponding data of each aiming field When table is multiple, then multiple tables of data are associated with to form target matrix, and first garbled data then derives from multiple tables of data.
When being screened again according to each aiming field to first garbled data, need according to corresponding to each aiming field Tables of data carry out;The tables of data corresponding to each aiming field is individual data table, and first garbled data is single from this When tables of data, then each aiming field of direct basis screens first garbled data again, obtains garbled data again.When each Tables of data corresponding to aiming field is multiple tables of data, when first garbled data derives from multiple tables of data, then according to each Each tables of data corresponding to a aiming field, screens each tables of data again, obtains garbled data again.Such as aiming field Including two class of insurer's name and insured amount, corresponding tables of data is respectively insurer's information table and insured amount table, sieve Select data of the range information between on December 31,1 day to 2018 July in 2018;Thus to according to demand field information, it is right Insurer's information table and insured amount table are screened for the first time, obtain time range in December 31 1 day to 2018 July in 2018 First garbled data between number.And then according to the corresponding relationship between each aiming field and each tables of data, insure to deriving from First garbled data in people's information table is screened again, obtains garbled data again corresponding with insurer's name;Simultaneously First garbled data in the insured amount table is screened again, obtains corresponding with insured amount screening number again According to.Because of the data that garbled data requires for demand field information in meet demand acquisition document with screening range information again, i.e., For the data of required acquisition;To which by this, garbled data is set as adopting with data corresponding to demand field information again Collection, with the acquisition demand of meet demand acquisition document.
In view of it is same to screen obtained garbled data again again when each target matrix is formed by multiple tables of data Sample derives from multiple tables of data;To this garbled data is acquired again when, need first to from multiple tables of data again Secondary garbled data is associated.Specifically, garbled data it data corresponding with demand field information will be set as will be acquired again The step of include:
Step S351, judges whether the target matrix carries the association identification, if carrying the association identification, According to the association identification, the garbled data again is associated with, and by garbled data is set as and institute again described in after association The corresponding data of demand field information are stated to be acquired;
Understandably, it when being associated the garbled data again from multiple tables of data, needs first to judge again Whether garbled data derives from multiple tables of data;It is associated if from multiple tables of data, otherwise without association.Because of mesh Tables of data is marked when being formed by multiple tables of data, each tables of data is assigned with association identification;So as to by association identification come Judge whether target matrix is formed by multiple tables of data, and then judges whether garbled data derives from multiple tables of data again. Specifically, judge whether target matrix carries association identification, illustrate target matrix by more if carrying association identification A tables of data is formed, and screens obtained first garbled data for the first time to target matrix from multiple tables of data, so that right First garbled data screens obtained garbled data again again and also derives from multiple tables of data.Multiple data are derived to this The garbled data again of table is associated according to association identification, i.e., distributes the garbled data again from each tables of data and close Connection mark, with the incidence relation established between each garbled data again;With according to each association identification to respectively again garbled data into The acquisition to required all data in demand acquisition document is realized in row acquisition.
The garbled data again is set as and the demand field by step S352 if not carrying the association identification The corresponding data of information are acquired.
Further, when judging that target matrix does not carry association identification, then illustrate target matrix by single Tables of data is formed, and screens obtained first garbled data for the first time to target matrix from individual data table, so as to first Secondary garbled data screens obtained garbled data again again and also derives from individual data table.It will be from individual data table Garbled data is directly set as data corresponding with demand field information again, and is acquired to the data, meet demand acquisition The acquisition demand of document.
The collecting method of the present embodiment, when receiving demand acquisition document, first to demand field information therein It is read out, recalls default foundation code, and demand field information is added in the connection sentence of default foundation code, it is raw At acquisition code;Hereafter control acquisition code operation generates operation result, and according to operation result determination and demand field information Corresponding aiming field, and tables of data corresponding with each aiming field;And then according to the quantity of tables of data, tables of data is formed Target matrix, and target matrix is screened, it realizes and data corresponding with demand field information is acquired.This programme will The data for needing to acquire form demand acquisition document, and demand field information therein characterizes the required data class acquired Type;Demand field information is added to acquisition code formed in default foundation code to be used to look into demand field information It looks for, to determine the tables of data where demand field information;And then screen tables of data formation target matrix, it acquires and needs Seek the corresponding data of field information.Because target matrix is formed as the tables of data where the demand field information for needing to acquire, keep away Exempt to save resource to the association and search operation of the tables of data unrelated with acquisition data, has improved the efficiency of data acquisition; Acquisition code is formed by by default foundation code simultaneously, lookup only is compared to demand field information at runtime, therefore is right The data volume of ratio is small, and search speed is fast.
Further, in another embodiment of collecting method of the present invention, the control acquisition code operation is raw Include: at before the step of operation result
Step S40 compares the acquisition code and presets rule, judges that each sentence in the acquisition code is It is no to meet the presets rule, wherein the presets rule is used to judge the correctness of the acquisition code;
Understandably, acquisition code needs in the correct situation of logic rules, could run and generate corresponding operation As a result.In order to ensure acquiring the correctness of code logic rule, the present embodiment is previously provided with the pre- of the correct logic rules of characterization If formal rule;The presets rule includes for being standardized to character boundary is write, symbol uses etc. in acquisition code Multinomial sub-rule such as uses the sub-rule standardized to symbol are as follows: and the symbol in the presence of each code statement individually occurs, The symbol continuously occurred is abnormal code statement.Control acquisition code operation before, will acquire code in each sentence by One and presets rule comparison, judge acquire code in each sentence whether meet preset rules.
Step S50 executes the control acquisition code fortune if each sentence meets the presets rule The step of row, generation operation result;
Step S60, if there is the abnormal statement for not meeting the presets rule in each sentence, based on described Abnormal statement output prompt modification information.
Further, if determining by contrast, each sentence meets preset rules, is illustrating the logic rules of acquisition code just Really, acquisition code operation is controlled, to generate operation result.And if determining to exist in each sentence by contrast and not meeting presets The sentence of rule, incongruent sentence are the abnormal statement of logic rules exception, need to be modified;By the abnormal statement institute Code line number be added to prompt modification information in export, to prompt developer to modify the abnormal statement.It is adopting Abnormal statement in the presence of each sentence of collection code is modified, and each sentence acquired in code is regular with presets After correspondence, control acquisition code operation generates operation result;To determine aiming field and and aiming field by operation result Corresponding tables of data, and then tables of data is formed into target matrix, therefrom acquire the number acquired required for demand acquisition document According to.
In addition, referring to figure 2., the present invention provides a kind of data acquisition device, it is real in data acquisition device first of the present invention It applies in example, the data acquisition device includes:
Read module 10, for reading the demand word in the demand acquisition document when receiving demand acquisition document Segment information, and default foundation code is called, the demand field information is added to the connection sentence of the default foundation code In, generate acquisition code;
Control module 20 generates operation result, and true according to the operation result for controlling the acquisition code operation Fixed aiming field corresponding with the demand field information, and tables of data corresponding with each aiming field;
The tables of data is formed target matrix, and to institute for the quantity according to the tables of data by acquisition module 30 It states target matrix to be screened, acquires data corresponding with the demand field information.
The data acquisition device of the present embodiment, when receiving demand acquisition document, read module 10 is first to need therein It asks field information to be read out, recalls default foundation code, and demand field information is added to the company of default foundation code It connects in sentence, generates acquisition code;Hereafter the control of control module 20 acquisition code operation, generates operation result, and according to operation As a result aiming field corresponding with demand field information, and tables of data corresponding with each aiming field are determined;And then acquire mould Tables of data is formed target matrix, and screen to target matrix according to the quantity of tables of data by block 30, realize to need Ask the corresponding data acquisition of field information.The data for needing to acquire are formed demand acquisition document, and demand therein by this programme The data type that field information acquires required for characterizing;Demand field information is added to formed in default foundation code Acquisition code is for searching demand field information, to determine the tables of data where demand field information;In turn by data Table forms target matrix and is screened, and acquires data corresponding with demand field information.Because target matrix is by needing to acquire Demand field information where tables of data formed, avoid and grasped to the association of the unrelated tables of data of acquisition data and searching Make, save resource, improves the efficiency of data acquisition;Acquisition code is formed by runtime by default foundation code simultaneously Lookup only is compared to demand field information, therefore the data volume compared is small, search speed is fast.
Further, in another embodiment of data acquisition device of the present invention, the control module further include:
Reading unit is complete in demand field information described in the operation result and the default foundation code for reading The comparing result of field comparison, and according to the comparing result, determine that demand field information described in the full word Duan Zhongyu is corresponding Aiming field;
First determination unit, for reading the corresponding data table name of each aiming field in the operation result, and root According to the data table name, determining tables of data corresponding with each aiming field.
Further, in another embodiment of data acquisition device of the present invention, the acquisition module further include:
Judging unit, for judging whether the corresponding data table name of each aiming field has uniqueness, if having only One property, then determine the tables of data quantity be it is single, otherwise determine the tables of data quantity be it is multiple;
Second determination unit, for when the quantity of the tables of data is single, being then determined as the single tables of data Target matrix;
Associative cell, for when the quantity of the tables of data is multiple, then reading in multiple tables of data and storing respectively The correlated identities of class data dependence, and the correlated identities are generated as association identification, multiple tables of data are associated with, shape At target matrix.
Further, in another embodiment of data acquisition device of the present invention, the acquisition module further include:
Generation unit, for reading the screening range information in the demand acquisition document, and according to the screening range Information screens the target matrix for the first time, generates first garbled data;
Acquisition unit, for according to tables of data corresponding with each aiming field, to each first garbled data into Row screens again, generates garbled data again, and the garbled data again is set as corresponding with the demand field information Data are acquired.
Further, in another embodiment of data acquisition device of the present invention, the acquisition unit is also used to:
Judge whether the target matrix carries the association identification, if the association identification is carried, according to The garbled data again is associated with by association identification, and by garbled data is set as and the demand word again described in after association The corresponding data of segment information are acquired;
If not carrying the association identification, the garbled data again is set as corresponding with the demand field information Data are acquired.
Further, in another embodiment of data acquisition device of the present invention, the read module further include:
Detection unit, for determining the connection sentence in the default foundation code according to connection identifier is preset, and according to Design variables mark detects the variable position in the connection sentence;
Adding unit replaces the variable position for the demand field information to be added in the variable position In variable, and detect whether the demand field information is added in the variable position;
Updating unit, if being updated to acquire by the default foundation code for being added in the variable position Code.
Further, in another embodiment of data acquisition device of the present invention, the data acquisition device further include:
Contrast module judges each in the acquisition code for comparing the acquisition code and presets rule Whether sentence meets the presets rule, wherein the presets rule is used to judge the correct of the acquisition code Property;
Execution module executes the control acquisition generation if meeting the presets rule for each sentence The step of code operation, generation operation result;
Output module, if for there is the abnormal statement for not meeting the presets rule, base in each sentence Prompt modification information is exported in the abnormal statement.
Wherein, each virtual functions module of above-mentioned data acquisition device is stored in the storage of data acquisition equipment shown in Fig. 3 In device 1005, when processor 1001 executes data acquisition program, the function of modules in embodiment illustrated in fig. 2 is realized.
Referring to Fig. 3, Fig. 3 is the device structure schematic diagram for the hardware running environment that present invention method is related to.
Data acquisition equipment of the embodiment of the present invention can be PC (personal computer, personal computer), can also be with It is the terminal devices such as smart phone, tablet computer, E-book reader, portable computer.
As shown in figure 3, the data acquisition equipment may include: processor 1001, such as CPU (Central Processing Unit, central processing unit), memory 1005, communication bus 1002.Wherein, communication bus 1002 for realizing Connection communication between processor 1001 and memory 1005.Memory 1005 can be high-speed RAM (random access Memory, random access memory), it is also possible to stable memory (non-volatile memory), such as disk storage Device.Memory 1005 optionally can also be the storage device independently of aforementioned processor 1001.
Optionally, which can also include user interface, network interface, camera, RF (Radio Frequency, radio frequency) circuit, sensor, voicefrequency circuit, WiFi (Wireless Fidelity, WiMAX) module etc.. User interface may include display screen (Display), input unit such as keyboard (Keyboard), and optional user interface can be with Including standard wireline interface and wireless interface.Network interface optionally may include standard wireline interface and wireless interface (such as WI-FI interface).
It will be understood by those skilled in the art that structure paired data does not acquire data acquisition equipment structure shown in Fig. 3 The restriction of equipment may include perhaps combining certain components or different component cloth than illustrating more or fewer components It sets.
As shown in figure 3, as may include operating system, net in a kind of memory 1005 of computer readable storage medium Network communication module and data acquisition program.Operating system is to manage and control the journey of data acquisition equipment hardware and software resource Sequence supports the operation of data acquisition program and other softwares and/or program.Network communication module is for realizing memory 1005 Communication between internal each component, and communicated between hardware and softwares other in data acquisition equipment.
In data acquisition equipment shown in Fig. 3, processor 1001 is adopted for executing the data stored in memory 1005 Collect program, realizes the step in each embodiment of above-mentioned collecting method.
The present invention provides a kind of computer readable storage medium, the computer-readable recording medium storage have one or More than one program of person, the one or more programs can also be executed by one or more than one processor with Step in the above-mentioned each embodiment of collecting method of realization.
It should also be noted that, herein, the terms "include", "comprise" or its any other variant are intended to non- It is exclusive to include, so that the process, method, article or the device that include a series of elements not only include those elements, It but also including other elements that are not explicitly listed, or further include solid by this process, method, article or device Some elements.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including There is also other identical elements in the process, method of the element, article or device.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art The part contributed out can be embodied in the form of software products, which is stored in one as described above In computer readable storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can To be mobile phone, computer, server or the network equipment etc.) execute method described in each embodiment of the present invention.
The above description is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all at this Under the design of invention, using equivalent structure transformation made by description of the invention and accompanying drawing content, or directly/it is used in it indirectly He is included in scope of patent protection of the invention relevant technical field.

Claims (10)

1. a kind of collecting method, which is characterized in that the collecting method the following steps are included:
When receiving demand acquisition document, the demand field information in the demand acquisition document is read, and call default base The demand field information is added in the connection sentence of the default foundation code by plinth code, generates acquisition code;
The acquisition code operation is controlled, generates operation result, and believe with the demand field according to the operation result is determining Cease corresponding aiming field, and tables of data corresponding with each aiming field;
According to the quantity of the tables of data, the tables of data is formed into target matrix, and sieve to the target matrix Choosing acquires data corresponding with the demand field information.
2. collecting method as described in claim 1, which is characterized in that it is described according to the operation result it is determining with it is described The corresponding aiming field of demand field information, and the step of tables of data corresponding with each aiming field includes:
Read the comparison knot of full word section comparison in demand field information described in the operation result and the default foundation code Fruit, and according to the comparing result, determine the corresponding aiming field of demand field information described in the full word Duan Zhongyu;
Read the corresponding data table name of each aiming field in the operation result, and according to the data table name, determine with The corresponding tables of data of each aiming field.
3. collecting method as claimed in claim 2, which is characterized in that the quantity according to the tables of data, by institute Stating the step of tables of data forms target matrix includes:
Judge whether the corresponding data table name of each aiming field has uniqueness, if having uniqueness, determines the number Quantity according to table be it is single, otherwise determine the tables of data quantity be it is multiple;
When the quantity of the tables of data is single, then the single tables of data is determined as target matrix;
When the quantity of the tables of data is multiple, then the correlation that Various types of data correlation is stored in multiple tables of data is read Mark, and the correlated identities are generated as association identification, multiple tables of data are associated with, target matrix is formed.
4. collecting method as claimed in claim 3, which is characterized in that it is described that the target matrix is screened, Acquire data corresponding with the demand field information the step of include:
The screening range information in the demand acquisition document is read, and according to the screening range information to the target data Table is screened for the first time, generates first garbled data;
According to tables of data corresponding with each aiming field, each first garbled data is screened again, is generated again Secondary garbled data, and the garbled data again is set as data corresponding with the demand field information and is acquired.
5. collecting method as claimed in claim 4, which is characterized in that described to be set as the garbled data again and institute Stating the step of corresponding data of demand field information are acquired includes:
Judge whether the target matrix carries the association identification, if carrying the association identification, according to the association The garbled data again is associated with by mark, and by garbled data is set as again and the demand field is believed described in after association Corresponding data are ceased to be acquired;
If not carrying the association identification, the garbled data again is set as data corresponding with the demand field information It is acquired.
6. collecting method as described in any one in claim 1-5, which is characterized in that described by the demand field information It is added in the connection sentence in the default foundation code, generating the step of acquiring code includes:
According to default connection identifier, the connection sentence in the default foundation code is determined, and identified according to design variables, detect Variable position in the connection sentence;
The demand field information is added in the variable position, the variable in the variable position is replaced, and detects institute Whether the demand field information of stating is added in the variable position;
If being added in the variable position, the default foundation code is updated to acquisition code.
7. collecting method as claimed in claim 6, which is characterized in that the control acquisition code operation generates Include: before the step of operation result
The acquisition code and presets rule are compared, it is described pre- to judge whether each sentence in the acquisition code meets If formal rule, wherein the presets rule is used to judge the correctness of the acquisition code;
If each sentence meets the presets rule, the control acquisition code operation is executed, operation knot is generated The step of fruit;
If there is the abnormal statement for not meeting the presets rule in each sentence, exported based on the abnormal statement Prompt modification information.
8. a kind of data acquisition device, which is characterized in that the data acquisition device includes:
Read module, for reading the demand field information in the demand acquisition document when receiving demand acquisition document, And default foundation code is called, the demand field information is added in the connection sentence of the default foundation code, is generated Acquire code;
Control module generates operation result, and according to the operation result determination and institute for controlling the acquisition code operation State the corresponding aiming field of demand field information, and tables of data corresponding with each aiming field;
The tables of data is formed target matrix, and to the target for the quantity according to the tables of data by acquisition module Tables of data is screened, and data corresponding with the demand field information are acquired.
9. a kind of data acquisition equipment, which is characterized in that the data acquisition equipment includes: memory, processor, communication bus And it is stored in the data acquisition program on the memory;
The communication bus is for realizing the connection communication between processor and memory;
The processor is for executing the data acquisition program, to realize such as data of any of claims 1-7 The step of acquisition method.
10. a kind of computer readable storage medium, which is characterized in that be stored with data on the computer readable storage medium and adopt Collect program, realizes that data of any of claims 1-7 such as acquire when the data acquisition program is executed by processor The step of method.
CN201910761596.XA 2019-08-15 2019-08-15 Collecting method, device, equipment and computer readable storage medium Pending CN110502513A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910761596.XA CN110502513A (en) 2019-08-15 2019-08-15 Collecting method, device, equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910761596.XA CN110502513A (en) 2019-08-15 2019-08-15 Collecting method, device, equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN110502513A true CN110502513A (en) 2019-11-26

Family

ID=68588207

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910761596.XA Pending CN110502513A (en) 2019-08-15 2019-08-15 Collecting method, device, equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN110502513A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112328223A (en) * 2020-11-27 2021-02-05 成都精灵云科技有限公司 Integrated machine platform
CN112948382A (en) * 2021-02-26 2021-06-11 平安科技(深圳)有限公司 Information processing method and device based on big data and related equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109656980A (en) * 2018-12-27 2019-04-19 Oppo(重庆)智能科技有限公司 Data processing method, electronic equipment, device and readable storage medium storing program for executing
CN110019486A (en) * 2018-07-19 2019-07-16 平安科技(深圳)有限公司 Collecting method, device, equipment and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110019486A (en) * 2018-07-19 2019-07-16 平安科技(深圳)有限公司 Collecting method, device, equipment and storage medium
CN109656980A (en) * 2018-12-27 2019-04-19 Oppo(重庆)智能科技有限公司 Data processing method, electronic equipment, device and readable storage medium storing program for executing

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112328223A (en) * 2020-11-27 2021-02-05 成都精灵云科技有限公司 Integrated machine platform
CN112948382A (en) * 2021-02-26 2021-06-11 平安科技(深圳)有限公司 Information processing method and device based on big data and related equipment

Similar Documents

Publication Publication Date Title
CN107894952A (en) Generation method, device, equipment and the readable storage medium storing program for executing of interface testing use-case
CN106055464B (en) Data buffer storage testing schooling pressure device and method
CN109634996A (en) Customer information table generating method, device, equipment and computer readable storage medium
CN105051729A (en) Data records selection
CN109376139A (en) Centralized database monitoring method, computer installation and storage medium
CN108388509B (en) Software testing method, computer readable storage medium and terminal equipment
CN110162566A (en) Association analysis method, device, computer equipment and the storage medium of business datum
CN106155884A (en) A kind of log analysis method and system
CN109726388A (en) Pdf document analytic method, device, equipment and computer readable storage medium
CN110502515A (en) Collecting method, device, equipment and computer readable storage medium
CN110362453A (en) Log statistic alarm method and device, terminal and storage medium
CN110321383A (en) Big data platform method of data synchronization, device, computer equipment and storage medium
CN108804484A (en) The data measures and procedures for the examination and approval, equipment and computer readable storage medium
CN114490375B (en) Performance test method, device, equipment and storage medium of application program
CN110502513A (en) Collecting method, device, equipment and computer readable storage medium
CN115236260A (en) Chromatographic data storage method and device, electronic equipment and storage medium
CN110515755A (en) Interface function verification method, device, equipment and computer readable storage medium
CN109412839A (en) A kind of recognition methods, device, equipment and the storage medium of exception account
CN108280096A (en) Data cleaning method and data cleansing device
CN109753555A (en) Word match method, apparatus, equipment and computer readable storage medium
CN110502545A (en) Date storage method, device, equipment and computer readable storage medium
CN102930016B (en) A kind of method and apparatus for providing Search Results on mobile terminals
CN110515924A (en) Database manipulation logic verify method, apparatus, equipment and readable storage medium storing program for executing
CN104933096B (en) Abnormal key recognition methods, device and the data system of database
CN109684213A (en) A kind of test method, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination