CN116244328A - Data acquisition method, device, equipment and storage medium - Google Patents
Data acquisition method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116244328A CN116244328A CN202310267081.0A CN202310267081A CN116244328A CN 116244328 A CN116244328 A CN 116244328A CN 202310267081 A CN202310267081 A CN 202310267081A CN 116244328 A CN116244328 A CN 116244328A
- Authority
- CN
- China
- Prior art keywords
- data
- user
- execution result
- data acquisition
- response
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data processing, and particularly discloses a data acquisition method, a device, equipment and a storage medium. The method comprises the following steps: when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request; establishing an association relation between preset data tables to obtain an association table; when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field; executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request. By the method, the SQL sentence can be automatically generated, the SQL sentence is executed to acquire multidimensional data, a user who does not need to acquire the data has the capability of writing the SQL sentence, the difficulty and the period of data acquisition are reduced, and meanwhile, the development cost is reduced.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data acquisition method, apparatus, device, and storage medium.
Background
The traditional data acquisition service needs to provide a section of complex SQL, and data can be acquired after the complex SQL is submitted to a database engineer for execution, so that in practical application, a user is required to have certain SQL capability, namely in the data acquisition process, SQL sentences are required to be written according to acquired contents, and then the SQL sentences are utilized for data acquisition. However, the user who collects data may not have the ability to write the SQL statement, so that the user cannot collect the data in time or needs to spend time and cost for SQL learning, which increases the data collection period and difficulty.
Disclosure of Invention
The invention provides a data acquisition method, a device, equipment and a storage medium, which can realize multi-dimensional data acquisition, reduce the difficulty and period of data acquisition and reduce the development cost.
In order to solve the technical problems, the invention adopts a technical scheme that: provided is a data acquisition method, comprising:
when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request;
establishing an association relation between the preset data tables to obtain an association table;
when receiving the data preview request input by the user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field;
executing the SQL sentence, responding to the data preview request to display an execution result, and responding to the data acquisition request to return the execution result to the user.
According to an embodiment of the present invention, when receiving the data preview request input by the user, obtaining the field selected by the user in each preset data table, and generating the SQL statement according to the association table and the field further includes:
when receiving the data preview request input by the user, acquiring fields selected by the user in each preset data table;
identifying a data type of the field;
configuring screening conditions according to the data types;
and generating an SQL statement according to the association table, the fields and the screening conditions.
According to one embodiment of the present invention, the executing the SQL statement, presenting an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request includes:
executing the SQL sentence to obtain an execution result;
acquiring the data record number according to the execution result, and judging whether the data record number exceeds a preset value;
if yes, screening the execution result, displaying the data record number and the screening result in response to the data preview request, and returning the data record number and the execution result to the user in response to the data acquisition request;
and if not, responding to the data preview request to display the data record number and the execution result, and responding to the data acquisition request to return the data record number and the execution result to the user.
According to one embodiment of the present invention, the filtering the execution result, and displaying the data record number and the filtering result in response to the data preview request includes:
screening the execution result;
identifying sensitive fields in the screening processing results and performing desensitization processing on the sensitive fields;
and responding to the data preview request, and displaying the data record number and the screening processing result after the desensitization processing.
According to one embodiment of the present invention, the executing the SQL statement, obtaining an execution result includes:
analyzing the SQL sentence to obtain an analysis result, wherein the analysis result comprises a table name of the preset data table where the target data corresponding to each field is located and a data type of the target data corresponding to each field;
and acquiring the target data from the preset data table based on the analysis result, the association table and the screening condition to obtain the execution result.
According to one embodiment of the present invention, before the executing the SQL statement, the presenting the execution result in response to the data preview request, and the returning the execution result to the user in response to the data acquisition request, the method further includes:
judging whether fields in the SQL statement are sensitive data or not;
if yes, an approval request is initiated, an approval passing response is received, and the SQL sentence is executed.
According to one embodiment of the present invention, the returning the execution result to the user in response to the data acquisition request includes:
adding a password to the execution result to carry out encryption processing;
and respectively returning the encrypted execution result and the corresponding password to the user in a mail mode in response to the data acquisition request.
In order to solve the technical problems, the invention adopts another technical scheme that: there is provided a data acquisition device comprising:
the acquisition module is used for acquiring a corresponding preset data table according to the data acquisition request and acquiring a field selected by a user in each preset data table when receiving the data acquisition request input by the user;
the establishing module is used for establishing the association relation between the preset data tables to obtain an association table;
the generating module is used for generating SQL sentences according to the association table and the fields when receiving the data preview request input by the user;
and the execution module is used for executing the SQL sentence, responding to the data preview request, displaying an execution result, responding to the data acquisition request and returning the execution result to the user.
In order to solve the technical problems, the invention adopts a further technical scheme that: there is provided a computer device comprising: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the data acquisition method when executing the computer program.
In order to solve the technical problems, the invention adopts a further technical scheme that: there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described data acquisition method.
The beneficial effects of the invention are as follows: when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request; establishing an association relation between preset data tables to obtain an association table; when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field; executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request, so that the SQL sentence can be automatically generated, multidimensional data acquisition can be performed by executing the SQL sentence, the user who does not need to acquire the data has the capability of writing the SQL sentence, the difficulty and the period of data acquisition are reduced, and meanwhile, the development cost is reduced.
Drawings
FIG. 1 is a flow chart of a data acquisition method according to an embodiment of the invention;
FIG. 2 is a flow chart of a data acquisition method according to another embodiment of the invention;
fig. 3 is a flowchart of step S207 in the data acquisition method according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of a data acquisition device according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a data acquisition device according to another embodiment of the present invention;
FIG. 6 is a schematic diagram of a computer device according to an embodiment of the invention;
fig. 7 is a schematic structural view of a computer storage medium according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," and the like in this disclosure are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", and "a third" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise. All directional indications (such as up, down, left, right, front, back … …) in embodiments of the present invention are merely used to explain the relative positional relationship, movement, etc. between the components in a particular gesture (as shown in the drawings), and if the particular gesture changes, the directional indication changes accordingly. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
Fig. 1 is a flow chart of a data acquisition method according to an embodiment of the invention. It should be noted that, if there are substantially the same results, the method of the present invention is not limited to the flow sequence shown in fig. 1. As shown in fig. 1, the method comprises the steps of:
step S101: when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request.
In step S101, a multi-dimensional data table is built in advance before providing the data acquisition service, and a plurality of fields are stored in each data table. When a user needs to collect data, a data collection request is input, and when the system receives the data collection request input by the user, a corresponding preset data table is obtained according to the data collection request.
Step S102: and establishing an association relation between preset data tables to obtain an association table.
In step S102, after the association table is established, the data collection service may be provided, and the association table stores a plurality of fields, association relations among the fields, and table names of the data table where the fields are located. For example, table 1 is a fact table, such as an order table, and tables 2 and 3 are dimension tables, such as a commodity information table, and table 1 and table 2 are associated, and table 1 and table 3 are associated, where the association table includes fields of table 1, table 2 and table 3, and association relations between fields. As another example, where the table 1 field a1 is related to the table 2 field a2 and the table 1 field b1 is related to the table 3 field b2, the association relationship between the data tables may be configured as: SELECT table 1..table 2..table 3..from table 1LEFT JOIN table 2ON table 1.a1=table 2.a2 LEFT JOIN table 3ON table 1.b1=table 3.b2.
The existing data acquisition service is based on a large-width table, and a user acquires data based on the large-width table, however, when new data needs exist, the large-width table needs to be rebuilt, and as the data acquisition needs are continuously increased, the difficulty of data maintenance work is increased, and finally, the user satisfaction is reduced. In the embodiment, when the fields of the table 1, the table 2 or the table 3 are changed, the association table is correspondingly modified, the association table does not need to be re-established, the data maintenance cost is reduced, the probability of data error is reduced, and the user experience is improved.
Step S103: when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field.
In step S103, the data may be previewed before the data is collected, and when a data preview request input by a user is received, a field selected by the user in each preset data table is obtained, and an SQL statement is automatically generated according to the association table and the field. For example, table 1 is a fact table, such as an order table, and tables 2 and 3 are dimension tables, such as a commodity information table, wherein, table 1 field a1 is related to table 2 field a2, table 1 field b1 is related to table 3 field b2, and then the association relationship between the data tables may be configured as follows: SELECT table 1..table 2..table 3..from table 1LEFT JOIN table 2ON table 1.a1=table 2.a2 LEFT JOIN table 3ON table 1.b1=table 3.b2. Assuming that the user selects the f1 field in table 1, the f2 field in table 2, and the f3 field in table 3, the SQL statement may be expressed as follows: SELECT table 1.f1, table 2.f2, table 3.f3 FROM (SELECT table 1..table 2..x, table 3..x FROM table 1LEFT JOIN table 2ON table 1.a1=table 2.a2 LEFT JOIN table 3ON table 1.b1=table 3.b2). The embodiment can automatically generate the SQL sentence, the user does not need to write the SQL sentence when collecting data, the difficulty and the period of data collection are reduced, and the development cost is reduced.
Step S104: executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request.
In step S104, the SQL statement is parsed to obtain a parsing result, where the parsing result includes a table name of a preset data table where the target data corresponding to each field is located and a data type of the target data corresponding to each field; and acquiring target data from a preset data table based on the analysis result and the association table to obtain an execution result.
In one implementation, before executing the SQL statement, determining whether fields in the SQL statement are sensitive data; if yes, an approval request is initiated, an approval passing response is received, and the SQL sentence is executed. In the embodiment, the sensitive data such as an identity card, a mobile phone number and the like relate to the sensitive data, and data acquisition can be performed only through approval, so that user privacy can be protected, illegal acquisition of user privacy is avoided, and user experience is improved.
In another implementation embodiment, in the step of returning the execution result to the user in response to the data acquisition request, a password is added to the execution result to perform encryption processing; and responding to the data acquisition request, respectively returning the encrypted execution result and the corresponding password to the user in a mail mode, and carrying out encryption processing on the execution result to improve the data security and avoid data leakage.
According to the data acquisition method, when a data acquisition request input by a user is received, a corresponding preset data table is acquired according to the data acquisition request; establishing an association relation between preset data tables to obtain an association table; when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field; executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request, so that the SQL sentence can be automatically generated, multidimensional data acquisition can be performed by executing the SQL sentence, the user who does not need to acquire the data has the capability of writing the SQL sentence, the difficulty and the period of data acquisition are reduced, and meanwhile, the development cost is reduced.
Fig. 2 is a flow chart of a data acquisition method according to another embodiment of the invention. It should be noted that, if there are substantially the same results, the method of the present invention is not limited to the flow sequence shown in fig. 2. As shown in fig. 2, the method comprises the steps of:
step S201: when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request.
In this embodiment, step S201 in fig. 2 is similar to step S101 in fig. 1, and is not described here again for brevity.
Step S202: and establishing an association relation between preset data tables to obtain an association table.
In this embodiment, step S202 in fig. 2 is similar to step S102 in fig. 1, and is not described herein for brevity.
Step S203: when receiving a data preview request input by a user, acquiring fields selected by the user in each preset data table.
In step S203, the data may be previewed before the data is collected, and when a data preview request input by the user is received, fields selected by the user in each preset data table are acquired, for example, the f1 field in table 1, the f2 field in table 2, and the f3 field in table 3 are selected by the user.
Step S204: the data type of the field is identified.
In step S204, the data type includes a number, time, character string, and the like.
Step S205: and configuring screening conditions according to the data types.
In step S205, the filtering condition is a preset condition, which may be set by a user or may be set by a default by a computer, for example, the data type of the f1 field in table 1 is a number, the filtering condition is configured such that the f1 field in table 1 is greater than a preset value, the data type of the f2 field in table 2 is a character string, the filtering condition is configured such that the f2 field in table 2 is not null, the data type of the f3 field in table 3 is time, and the filtering condition is configured such that the format of the f3 field in table 3 is year/month/day.
Step S206: and generating the SQL statement according to the association table, the fields and the screening conditions.
In step S206, the SQL statement is automatically generated according to the association table, the fields and the filtering condition, for example, table 1 is a fact table, such as an order table, table 2 and table 3 are dimension tables, such as a commodity information table, wherein the table 1 field a1 is related to the table 2 field a2, the table 1 field b1 is related to the table 3 field b2, and then the association relationship between the data tables may be configured as follows: SELECT table 1..table 2..table 3..from table 1LEFT JOIN table 2ON table 1.a1=table 2.a2 LEFT JOIN table 3ON table 1.b1=table 3.b2. Assuming that the user selects the f1 field in table 1, the f2 field in table 2, and the f3 field in table 3, the SQL statement may be expressed as follows: SELECT table 1.f1, table 2.f2, table 3.f3 FROM (SELECT table 1. Table 2. Table 3. FROM table 1LEFT JOIN table 2ON table 1.a1=table 2.a2 LEFT JOIN table 3ON table 1.b1=table 3.b2) WHERE table 1.f1>20and table 2.f2 IS NOT NULL.
Step S207: executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request.
In step S207, the SQL statement is parsed to obtain a parsing result, an execution result is displayed in response to the data preview request, and the execution result is returned to the user in response to the data acquisition request.
In one implementation embodiment, in the step of returning the execution result to the user in response to the data acquisition request, encryption processing is performed by adding a password to the execution result; and responding to the data acquisition request, respectively returning the encrypted execution result and the corresponding password to the user in a mail mode, and carrying out encryption processing on the execution result to improve the data security and avoid data leakage.
Further, referring to fig. 3, step S207 further includes the following steps:
step S301: and executing the SQL sentence to obtain an execution result.
Specifically, analyzing the SQL sentence to obtain an analysis result, wherein the analysis result comprises a table name of a preset data table where the target data corresponding to each field is located and a data type of the target data corresponding to each field; and acquiring target data from a preset data table based on the analysis result, the association table and the screening condition to obtain an execution result.
Step S302: and acquiring the data record number according to the execution result, and judging whether the data record number exceeds a preset value.
Specifically, the number of data records is used to characterize the data amount of the execution result, and the preset value can be adjusted according to the actual situation, for example, the preset value is 1000.
Step S303: if yes, screening the execution result, displaying the data record number and the screening result in response to the data preview request, and returning the data record number and the execution result to the user in response to the data acquisition request.
In step S303, if the number of data records exceeds the preset value, it is indicated that the data size of the execution result is relatively large, and when previewing, the execution result is filtered to obtain a part of sample data, and the number of data records and the part of sample data are displayed in response to the data preview request, so that the data preview throughput is reduced, and the data acquisition efficiency is improved.
Specifically, in one implementation embodiment, the execution results are filtered; identifying sensitive fields in the screening processing results and performing desensitization processing on the sensitive fields; and responding to the data preview request, and displaying the data record number and the screening processing result after the desensitization processing. According to the embodiment, the sensitive fields in the sample data, such as the identity card number, the mobile phone number and the like, are subjected to desensitization treatment, so that the user privacy can be protected, the user privacy is prevented from being revealed, and the user experience is improved.
Step S304: and if not, displaying the data record number and the execution result in response to the data preview request, and returning the data record number and the execution result to the user in response to the data acquisition request.
In step S304, if the number of data records does not exceed the preset value, which indicates that the data size of the execution result is relatively small, all the execution results may be previewed during the preview.
The data acquisition method of the embodiment of the invention is based on the embodiment by identifying the data type of the field; configuring screening conditions according to the data types; and generating the SQL statement according to the association table, the fields and the screening conditions, so that the accuracy of the SQL statement can be improved, and the accuracy and the efficiency of data acquisition are ensured.
Fig. 4 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention. As shown in fig. 4, the apparatus 40 includes an acquisition module 41, a setup module 42, a generation module 43, and an execution module 44.
The acquiring module 41 is configured to acquire a corresponding preset data table according to a data acquisition request when receiving the data acquisition request input by a user;
the establishing module 42 is configured to establish an association relationship between preset data tables, and obtain an association table;
the generating module 43 is configured to obtain a field selected by a user in each preset data table when receiving a data preview request input by the user, and generate an SQL statement according to the association table and the field;
the execution module 44 is configured to execute the SQL statement, expose an execution result in response to the data preview request, and return the execution result to the user in response to the data acquisition request.
Further, in an implementation embodiment, referring to fig. 5, the generating module 43 further includes an obtaining unit 431, an identifying unit 432, a configuring unit 433, and a generating unit 434.
The acquiring unit 431 is configured to acquire a field selected by a user in each preset data table when receiving a data preview request input by the user.
The identifying unit 432 is used for identifying the data type of the field.
The configuration unit 433 is configured to configure the filtering condition according to the data type.
The generating unit 434 is configured to generate an SQL statement according to the association table, the fields, and the filtering conditions.
Further, in one implementation, referring to fig. 5, the execution module 44 further includes: the execution unit 441, the determination unit 442, the first response unit 443, and the second response unit 444.
The execution unit 441 is configured to execute an SQL statement to obtain an execution result.
The judging unit 442 is configured to obtain the number of data records according to the execution result, and judge whether the number of data records exceeds a preset value.
The first response unit 443 is configured to, if yes, perform filtering processing on the execution result, display the number of data records and the filtering processing result in response to the data preview request, and return the number of data records and the execution result to the user in response to the data acquisition request.
The second response unit 444 is configured to, if not, display the number of data records and the execution result in response to the data preview request, and return the number of data records and the execution result to the user in response to the data acquisition request.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the invention. As shown in fig. 6, the computer device 60 includes a processor 61 and a memory 62 coupled to the processor 61.
The memory 62 stores program instructions for implementing the data acquisition method described in any of the embodiments above.
The processor 61 is configured to execute program instructions stored in the memory 62 to collect data.
In one implementation embodiment, the data acquisition method includes:
when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request;
establishing an association relation between preset data tables to obtain an association table;
when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field;
executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request.
The processor 61 may also be referred to as a CPU (Central Processing Unit ). The processor 61 may be an integrated circuit chip with signal processing capabilities. Processor 61 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a computer storage medium according to an embodiment of the present invention. The computer storage medium according to the embodiment of the present invention stores a program file 71 capable of implementing all the methods described above, where the program file 71 may be stored in the form of a software product in the computer storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the embodiments of the present invention.
In one implementation embodiment, the data acquisition method includes:
when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request;
establishing an association relation between preset data tables to obtain an association table;
when receiving a data preview request input by a user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field;
executing the SQL sentence, displaying an execution result in response to the data preview request, and returning the execution result to the user in response to the data acquisition request.
And the aforementioned computer storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes, or a terminal device such as a computer, a server, a mobile phone, a tablet, or the like.
In the several embodiments provided in the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The foregoing is only the embodiments of the present invention, and therefore, the patent scope of the invention is not limited thereto, and all equivalent structures or equivalent processes using the descriptions of the present invention and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the invention.
Claims (10)
1.A method of data acquisition, comprising:
when receiving a data acquisition request input by a user, acquiring a corresponding preset data table according to the data acquisition request;
establishing an association relation between the preset data tables to obtain an association table;
when receiving the data preview request input by the user, acquiring a field selected by the user in each preset data table, and generating an SQL sentence according to the association table and the field;
executing the SQL sentence, responding to the data preview request to display an execution result, and responding to the data acquisition request to return the execution result to the user.
2. The method of claim 1, wherein when receiving the data preview request input by the user, obtaining the field selected by the user in each preset data table, and generating the SQL statement according to the association table and the field further comprises:
when receiving the data preview request input by the user, acquiring fields selected by the user in each preset data table;
identifying a data type of the field;
configuring screening conditions according to the data types;
and generating an SQL statement according to the association table, the fields and the screening conditions.
3. The data collection method according to claim 2, wherein executing the SQL statement, presenting an execution result in response to the data preview request, and returning the execution result to the user in response to the data collection request comprises:
executing the SQL sentence to obtain an execution result;
acquiring the data record number according to the execution result, and judging whether the data record number exceeds a preset value;
if yes, screening the execution result, displaying the data record number and the screening result in response to the data preview request, and returning the data record number and the execution result to the user in response to the data acquisition request;
and if not, responding to the data preview request to display the data record number and the execution result, and responding to the data acquisition request to return the data record number and the execution result to the user.
4. A data collection method according to claim 3, wherein said filtering the execution result, and displaying the number of data records and the result of the filtering in response to the data preview request comprises:
screening the execution result;
identifying sensitive fields in the screening processing results and performing desensitization processing on the sensitive fields;
and responding to the data preview request, and displaying the data record number and the screening processing result after the desensitization processing.
5. The data collection method according to claim 3, wherein executing the SQL statement to obtain an execution result comprises:
analyzing the SQL sentence to obtain an analysis result, wherein the analysis result comprises a table name of the preset data table where the target data corresponding to each field is located and a data type of the target data corresponding to each field;
and acquiring the target data from the preset data table based on the analysis result, the association table and the screening condition to obtain the execution result.
6. The data collection method according to claim 1, wherein the executing the SQL statement, presenting an execution result in response to the data preview request, and before returning the execution result to the user in response to the data collection request, further comprises:
judging whether fields in the SQL statement are sensitive data or not;
if yes, an approval request is initiated, an approval passing response is received, and the SQL sentence is executed.
7. The data acquisition method of claim 1 wherein the returning the execution result to the user in response to the data acquisition request comprises:
adding a password to the execution result to carry out encryption processing;
and respectively returning the encrypted execution result and the corresponding password to the user in a mail mode in response to the data acquisition request.
8. A data acquisition device, comprising:
the acquisition module is used for acquiring a corresponding preset data table according to the data acquisition request when receiving the data acquisition request input by a user;
the establishing module is used for establishing the association relation between the preset data tables to obtain an association table;
the generating module is used for acquiring fields selected by a user in each preset data table when receiving the data preview request input by the user, and generating SQL sentences according to the association table and the fields;
and the execution module is used for executing the SQL sentence, responding to the data preview request, displaying an execution result, responding to the data acquisition request and returning the execution result to the user.
9. A computer device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the data acquisition method according to any one of claims 1-7 when executing the computer program.
10. A computer storage medium having stored thereon a computer program, which when executed by a processor implements the data acquisition method according to any of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310267081.0A CN116244328A (en) | 2023-03-16 | 2023-03-16 | Data acquisition method, device, equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310267081.0A CN116244328A (en) | 2023-03-16 | 2023-03-16 | Data acquisition method, device, equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116244328A true CN116244328A (en) | 2023-06-09 |
Family
ID=86629566
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310267081.0A Pending CN116244328A (en) | 2023-03-16 | 2023-03-16 | Data acquisition method, device, equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116244328A (en) |
-
2023
- 2023-03-16 CN CN202310267081.0A patent/CN116244328A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10853564B2 (en) | Operation for copied content | |
CN110990274B (en) | Data processing method, device and system for generating test cases | |
CN109542764B (en) | Webpage automatic testing method and device, computer equipment and storage medium | |
CN112417274A (en) | Message pushing method and device, electronic equipment and storage medium | |
CN110058995B (en) | Database testing method and system capable of avoiding interference of database types | |
CN110019076B (en) | Method, device and equipment for constructing multi-system log data and readable storage medium | |
CN108509582B (en) | Information reply method, terminal equipment and computer readable storage medium | |
CN114138651A (en) | Test data generation method and device | |
CN104182479B (en) | A kind of method and device handling information | |
CN108228611B (en) | Document information copying method and device | |
KR20130126012A (en) | Method and apparatusfor providing report of business intelligence | |
CN111190965A (en) | Text data-based ad hoc relationship analysis system and method | |
CN111736825B (en) | Information display method, device, equipment and storage medium | |
CN117056352A (en) | Data display method, device, terminal equipment and readable storage medium | |
US20170364580A1 (en) | Information processing apparatus, information processing method, and non-transitory computer readable medium | |
CN116244328A (en) | Data acquisition method, device, equipment and storage medium | |
CN114741594A (en) | Information pushing method and device, computer equipment and storage medium | |
CN113420042A (en) | Data statistics method, device, equipment and storage medium based on presentation | |
CN108460159B (en) | Information reply method, terminal equipment and computer readable storage medium | |
CN111078668A (en) | Data generation method and device, electronic equipment and storage medium | |
CN116303627B (en) | Query method and device for semiconductor test data, electronic equipment and storage medium | |
TWM560616U (en) | An electronic device for providing an associated menu | |
CN111126737B (en) | Cross-scene cross analysis method and device, electronic equipment and storage medium | |
CN113822046B (en) | Analysis method, device, equipment and storage medium based on document test case | |
CN115659406B (en) | Data access method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |