CN115470229A - Data table processing method and device, electronic equipment and storage medium - Google Patents
Data table processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115470229A CN115470229A CN202211045460.7A CN202211045460A CN115470229A CN 115470229 A CN115470229 A CN 115470229A CN 202211045460 A CN202211045460 A CN 202211045460A CN 115470229 A CN115470229 A CN 115470229A
- Authority
- CN
- China
- Prior art keywords
- sql
- time
- data table
- processing
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a data table processing method and device, electronic equipment and a storage medium, and belongs to the technical field of internet. The method comprises the following steps: generating a time parameter according to the time dimension type and the time period; acquiring a first SQL file, wherein the first SQL file is used for executing a processing request and is formed by splicing a plurality of SQL sentences, and codes used for representing time in the SQL sentences are expressed by adopting time variables; replacing the time variable of each SQL statement included in the first SQL file with a time parameter to obtain a second SQL file; and writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table. According to the method and the device, the SQL sentences used for executing the same processing request are written into the same SQL file, and then the time variables in the SQL sentences are replaced in a unified mode, so that the problem that the time variables in the SQL sentences are not modified uniformly is solved, and the accuracy of the processed data table is improved.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for processing a data table, an electronic device, and a storage medium.
Background
In the field of internet technology, due to business requirements, calculation results of the same index in a data table at different time granularities of week, month, year and the like need to be obtained. Generally speaking, the computation logic of the same index at different time granularities is the same. In practical application, the calculation of the same index at different time granularities usually obtains an SQL file of the index at a certain time granularity, the SQL file comprises at least one SQL statement, then based on the time at the current time granularity, a technician manually modifies each SQL statement in the SQL file to obtain a modified SQL file, and further based on each SQL statement in the modified SQL file, calculates the index in a data table to obtain a processed data table.
However, the above method is prone to modification inconsistency, resulting in low accuracy of the processed data table.
Disclosure of Invention
The embodiment of the disclosure provides a data table processing method and device, electronic equipment and a storage medium, which can improve the accuracy of a processed data table. The technical scheme is as follows:
in a first aspect, a method for processing a data table is provided, where the method includes:
receiving a processing request aiming at a target index in a target data table, wherein the processing request comprises a time dimension type and a time period;
generating a time parameter according to the time dimension type and the time period;
acquiring a first SQL file, wherein the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables;
replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and writing a processing result generated by executing the second SQL file and aiming at the target index into the target data table.
In another embodiment of the present disclosure, before writing the processing result generated by executing the second SQL file and for the target index into the target data table, the method further includes
The SQL sentences included in the second SQL file are segmented to obtain a plurality of time-substituted SQL sentences;
executing each SQL statement after the time replacement to obtain a processing result corresponding to each SQL statement after the time replacement;
the writing, into the target data table, a processing result for the target index generated by executing the second SQL file includes:
and writing a corresponding processing result into the target database according to the type of each SQL statement after the time replacement.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
and when the type of the SQL statement after the time replacement is a writing type, writing a processing result corresponding to the SQL statement after the time replacement into the target database.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
when the type of the SQL statement after the time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before the processing result generated by executing the second SQL file and for the target indicator is written into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the two time fields corresponding to the target index are modified, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
In a second aspect, there is provided a data table processing apparatus, the apparatus comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a processing request aiming at a target index in a target data table, and the processing request comprises a time dimension type and a time period;
the generating module is used for generating a time parameter according to the time dimension type and the time period;
the acquisition module is used for acquiring a first SQL file, the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables;
the replacing module is used for replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and the writing module is used for writing the processing result which is generated by executing the second SQL file and aims at the target index into the target data table.
In another embodiment of the present disclosure, the apparatus further comprises
The segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
and the writing module is used for writing the corresponding processing result into the target database according to the type of each SQL statement after time replacement.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
and when the type of the SQL statement after the time replacement is a writing type, writing a processing result corresponding to the SQL statement after the time replacement into the target database.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
when the type of the SQL statement after the time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before writing the processing result generated by executing the second SQL file and directed to the target indicator into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the two time fields corresponding to the target index are modified, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
In a third aspect, an electronic device is provided, which includes a processor and a memory, where at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the data table processing method according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the data table processing method according to the first aspect.
In a fifth aspect, a computer program product is provided, the computer program product comprising computer program code stored in a computer-readable storage medium, the computer program code being read from the computer-readable storage medium by a processor of an electronic device, the processor executing the computer program code to cause the electronic device to perform the data table processing method according to the first aspect.
The technical scheme provided by the embodiment of the disclosure has the following beneficial effects:
the target indexes needing to be processed are written into the target data table, the SQL sentences used for executing the same processing request are written into the same SQL file, and when the processing result of the target indexes in a certain time period under a certain time dimension type needs to be obtained, the time variables of all the SQL sentences in the SQL file are replaced in a unified mode, the problem that the time variables in all the SQL sentences are not modified uniformly is solved, and the accuracy of the processed data table is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a flow chart of a method for processing a data table according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of another method for processing a data table provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart of another method for processing a data table provided by an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a data table processing apparatus according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
It is understood that, as used in the embodiments of the present disclosure, the terms "each," "a plurality," and "either," and the like, include two or more than two, each referring to each of the corresponding plurality and any referring to any one of the corresponding plurality. For example, the plurality of words includes 10 words, and each word refers to each of the 10 words, and any word refers to any one of the 10 words.
Information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals to which the present disclosure relates are authorized by the user or sufficiently authorized by various parties, and the collection, use, and processing of the relevant data requires compliance with relevant laws and regulations and standards in the relevant countries and regions.
The embodiment of the present disclosure provides a data table processing method, which is implemented by taking an electronic device as an example, where the electronic device has a relatively high computing capability, the electronic device may be a terminal, such as a notebook computer, a desktop computer, and the like, and the electronic device may also be a server, such as an individual physical server, a cluster or distributed system formed by multiple physical servers, and the like. Referring to fig. 1, a method flow provided by the embodiment of the present disclosure includes:
101. a processing request for a target metric in a target data table is received.
In the embodiment of the disclosure, when a processing result of a target index in a target data table needs to be obtained due to a service requirement, a user may send a processing request for the target index in the target data table to an electronic device. The target data table is a data table for bearing the target indexes and the processing results of the target indexes. The target data table includes a plurality of indexes, the index refers to a field name of a certain field in the target data table, and the index may be gender, age, salary and the like. The processing request is used for requesting the electronic equipment to process data related to the target indexes in the target database in the time period according to the time dimension type, the processing request comprises the time dimension type, the time period and the like, and the time dimension type comprises the types of year, month, week, day and the like.
102. And generating a time parameter according to the time dimension type and the time period.
In the embodiment of the present disclosure, the processing request received by the electronic device is actually an SQL statement, and the SQL statement needs to be executed by a scheduler, which is a program for executing an SQL file. In response to a received processing request for a target index in the target data table, the electronic device calls a scheduling program, and then generates a time parameter according to the time dimension type and the time period.
Specifically, the electronic device divides a time period according to a time dimension type to obtain a time parameter of the time period in the time dimension type. For example, if the time dimension type is week, and the time period is from 1 month 1 to 1 month 21, the electronic device divides the time period from 1 month 1 to 1 month 21 according to the week, and obtains the time parameters as: no. 1-No. 7, no. 1-No. 8-No. 1-No. 14, and No. 1-No. 14-No. 1-No. 21.
103. And acquiring a first SQL file.
In the embodiment of the present disclosure, according to different service types, the electronic device splices SQL statements that process the same service in advance, and places the spliced SQL statements in one SQL file. Usually, these SQL files are stored in a hard disk, and when a processing request for a target index in a target data table is received, in response to the processing request, the electronic device obtains a first SQL file corresponding to the service type from the hard disk, and further writes the first SQL file into a memory, thereby facilitating subsequent processing of the first SQL file. The first SQL file is used for executing a processing request aiming at a target index in a target data table and is formed by splicing a plurality of SQL sentences.
In the embodiment of the present disclosure, the code for representing time in the SQL statements included in the SQL file is represented by a time variable. Generally, the time variable names of different time dimension types are consistent, generally, only a few time variables are needed, and for special time, the time variable names can be represented by time functions. Because the code used for representing the time in each SQL statement included in the first SQL file is represented by a time variable and has no exact execution time, the electronic device cannot execute the first SQL file and needs to modify the time variable in the first SQL file to a time constant.
104. And replacing the time variable of each SQL statement included in the first SQL file with a time parameter to obtain a second SQL file.
Based on the acquired time variable, the electronic equipment identifies the time variable in each SQL statement in the first SQL file, and then replaces the time variable of each SQL statement with the time parameter in a unified manner to obtain a second SQL file. Because the code for representing time in each SQL statement in the second SQL file is changed from the time variable to the time constant, the electronic device can execute the second SQL file. By executing the second SQL file, the processing result of the target index can be obtained.
105. And writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table.
In the embodiment of the present disclosure, before writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table, the electronic device will perform the following operations:
the first step is that the electronic equipment divides SQL sentences included in the second SQL file to obtain a plurality of SQL sentences after time replacement.
In the embodiment of the present disclosure, because the SQL statements in the second SQL file are spliced together, and the operations executed by different SQL statements are different, in order to execute different operations based on different SQL statements, the electronic device needs to segment the SQL statements included in the second SQL file. When the electronic device splits the second SQL file, a symbol (for example, an english comma) may be preset as a splitting point to split the second SQL file. And segmenting the second SQL file to obtain a plurality of time-replaced SQL sentences, wherein the plurality of time-replaced SQL sentences are independent from each other and can be executed independently.
And secondly, the electronic equipment executes each time-replaced SQL statement to obtain a processing result corresponding to each time-replaced SQL statement.
Based on each time-replaced SQL statement, the electronic device can obtain a processing result corresponding to each time-replaced SQL statement by executing each time-replaced SQL statement, and then writes the processing result aiming at the target index generated by executing the second SQL file into the target data table.
In the disclosed embodiments, the SQL statements are of different types, including a write type and a view type. Generally, a write type SQL statement begins with Insert and a view type SQL statement begins with @. Therefore, different types of SQL sentences can be distinguished according to the initial characters of the different types of SQL sentences. For example, for any SQL statement, @ v _ org select org _ id, org _ name form i _ org where dt = '2022-02-22', the type of the SQL statement can be identified as the view type according to the initial character of the SQL statement.
Because the operations executed by different types of SQL statements are different, the forms of the obtained processing results are also different, and the processing modes of the electronic device are different for different forms of processing results, the electronic device can write the corresponding processing results into the target database in different modes according to the types of the SQL statements replaced at each time.
In a possible implementation manner, when the type of the SQL statement after the time replacement is the write type, the electronic device may directly write the processing result corresponding to the SQL statement after the time replacement into the target database.
In another possible implementation manner, when the type of the SQL statement after time replacement is a view type, the processing result corresponding to the SQL statement after time replacement is generally an intermediate processing result, and the SQL statement after time replacement often needs to read the intermediate processing result to generate a processing result corresponding to the SQL statement after time transformation, so that the electronic device does not directly write the processing result corresponding to the SQL statement after time replacement into the target data table, but generates a temporary view based on the processing result corresponding to the SQL statement after time replacement. The data in the temporary view is the execution result of the temporally replaced SQL statement, for example, if the temporally replaced SQL statement is "select org _ id, org _ name from i _ org where dt = '2022-02-22'", then the data in the temporary view is the execution result of "select org _ id, org _ name from i _ org where dt = '2022-02-22'". Because the temporary view is stored in the memory and does not need to be displayed and created, the subsequent table building and maintenance work is saved, and the processing efficiency of the data table is improved.
Further, when the SQL statements replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL statements replaced at other times, the electronic device writes the processing results corresponding to the SQL statements replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, where the time dimension type field is used to distinguish different time granularities, and tinyint types may be used; the time period field can adopt a date type, and is convenient for partitioning different databases.
In another embodiment of the present disclosure, to avoid confusion of processing results of different time granularities corresponding to the target index, the electronic device further modifies the time dimension type field corresponding to the target index into a time dimension type and modifies the time period field into a time period before writing the processing result for the target index generated by executing the second SQL file into the target data table, so that in a case where modification of two time fields corresponding to the target index is completed, an operation of writing the processing result for the target index generated by the second SQL file into the target data table is executed.
In another embodiment of the disclosure, in response to a processing request for a target index in a target data table, when the electronic device executes a processing process for the target index, an operation involved in the processing process is written into a log corresponding to the target data table. When the abnormal record log is identified, the electronic equipment also sends the abnormal content and the context to a technician so that the technician can trace the processing process conveniently. When the electronic device sends the abnormal content to the technician, the abnormal content can be sent by adopting a network, a mail, an instant messaging tool and the like, and the embodiment of the disclosure does not limit the sending mode of the abnormal content.
By adopting the method provided by the embodiment of the disclosure, a set of SQL codes can be used, the calculation of the same index of the time granularity of a plurality of days, weeks, months and the like can be realized, and compared with the mode that the time codes in the SQL sentences need to be manually modified one by one in the related art, the method is more convenient to debug, and is more convenient to change particularly when bugs or business changes are repaired. In addition, the generated temporary view does not need to be displayed and maintained, and is friendly to both development and execution.
Fig. 2 shows an overall flow of a data table processing method provided by the embodiment of the present disclosure, and referring to fig. 3, the processing flow includes the following steps:
1. when a processing request aiming at a target index in a target data table is received, the electronic equipment calls a scheduling program, and the scheduling program generates time corresponding to a time variable;
2. the scheduling degree reads the SQL file;
3. the scheduling program adopts specific time to replace time variables of all SQL sentences in the SQL file;
4. the processed SQL file is divided into single SQL sentences, and the SQL sentences are analyzed and recombined according to different types;
5. and writing the processing result into the target data table by executing the split SQL sentence.
Fig. 3 shows different processing procedures based on different types of SQL statements, and referring to fig. 3, the electronic device determines the type of the SQL statement from the symbol at the beginning of the SQL statement. When the type of the SQL statement is a writing type, writing a processing result of the SQL statement into a target data table; and when the type of the SQL statement is the view type, generating a Spark temporary view, and further writing the processing result of other SQL statements based on the temporary view into the target data table.
According to the method provided by the embodiment of the disclosure, the target indexes to be processed are written into the target data table, the SQL sentences used for executing the same processing request are written into the same SQL file, and when the processing result of the target indexes in a certain time period under a certain time dimension type needs to be obtained, the time variables of all the SQL sentences in the SQL file are replaced in a unified manner, so that the problem that the time variables in all the SQL sentences are not modified uniformly is solved, and the accuracy of the processed data table is improved.
Referring to fig. 4, an embodiment of the present disclosure provides a data table processing apparatus, including:
a receiving module 401, configured to receive a processing request for a target indicator in a target data table, where the processing request includes a time dimension type and a time period;
a generating module 402, configured to generate a time parameter according to the time dimension type and the time period;
an obtaining module 403, configured to obtain a first SQL file, where the first SQL file is used to execute a processing request, the first SQL file is formed by splicing a plurality of SQL statements, and a code in the SQL statement for representing time is represented by a time variable;
a replacing module 404, configured to replace a time variable of each SQL statement included in the first SQL file with a time parameter, to obtain a second SQL file;
and a writing module 405, configured to write a processing result generated by executing the second SQL file and for the target index into the target data table.
In another embodiment of the present disclosure, the apparatus further comprises
The segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each time-replaced SQL statement to obtain a processing result corresponding to each time-replaced SQL statement;
and the writing module is used for writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time.
In another embodiment of the present disclosure, writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time includes:
and when the type of the SQL statement after time replacement is a write-in type, writing a processing result corresponding to the SQL statement after time replacement into the target database.
In another embodiment of the present disclosure, writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time includes:
when the type of the SQL statement after time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before writing a processing result generated by executing the second SQL file and directed to the target indicator into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into a time dimension type, and modifying a time period field into a time period;
and under the condition that the modification of the two time fields corresponding to the target index is completed, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
To sum up, the apparatus provided in this embodiment of the present disclosure writes a target index to be processed into a target data table, writes an SQL statement for executing a same processing request into a same SQL file, and when a processing result of the target index in a certain time period under a certain time dimension type needs to be obtained, performs unified replacement on a time variable of each SQL statement in the SQL file, overcomes a problem that the time variable in each SQL statement is not uniformly modified, and improves accuracy of the processed data table.
Fig. 5 shows a block diagram of an electronic device 500 according to an exemplary embodiment of the present disclosure. In general, the electronic device 500 includes: a processor 501 and a memory 502.
The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in a wake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 501 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
In some embodiments, the electronic device 500 may further optionally include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502 and peripheral interface 503 may be connected by a bus or signal lines. Each peripheral may be connected to the peripheral interface 503 by a bus, signal line, or circuit board. Specifically, the peripheral device includes: a power supply 504.
The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on separate chips or circuit boards, which is not limited by the present embodiment.
The power supply 504 is used to supply power to the various components in the electronic device 500. The power source 504 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power supply 504 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is not intended to be limiting of the electronic device 500 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of the electronic device 500 to perform the database processing method described above is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium may be a CD-ROM (Compact Disc Read-Only Memory), a ROM, a RAM (Random Access Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.
The embodiment of the disclosure provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the above data table processing method.
The disclosed embodiments provide a computer program product, which includes a computer program code, the computer program code being stored in a computer-readable storage medium, the computer program code being read from the computer-readable storage medium by a processor of an electronic device, and the computer program code being executed by the processor, so that the electronic device executes the above data table processing method.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk.
The above description is intended to be exemplary only and not to limit the present disclosure, and any modification, equivalent replacement, or improvement made without departing from the spirit and scope of the present disclosure is to be considered as the same as the present disclosure.
Claims (10)
1. A method of data table processing, the method comprising:
receiving a processing request aiming at a target index in a target data table, wherein the processing request comprises a time dimension type and a time period;
generating a time parameter according to the time dimension type and the time period;
acquiring a first SQL file, wherein the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables;
replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and writing a processing result generated by executing the second SQL file and aiming at the target index into the target data table.
2. The method according to claim 1, wherein before writing the processing result generated by executing the second SQL file for the target indicator into the target data table, further comprising:
the SQL sentences included in the second SQL file are segmented to obtain a plurality of time-replaced SQL sentences;
executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
the writing, into the target data table, a processing result for the target index generated by executing the second SQL file includes:
and writing the corresponding processing result into the target database according to the type of each SQL statement after time replacement.
3. The method according to claim 2, wherein writing the corresponding processing result into the target database according to the type of each time-substituted SQL statement comprises:
and when the type of the SQL statement after the time replacement is a writing type, writing a processing result corresponding to the SQL statement after the time replacement into the target database.
4. The method according to claim 2, wherein writing the corresponding processing result into the target database according to the type of each time-substituted SQL statement comprises:
when the type of the SQL statement after the time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
5. The method according to any one of claims 1 to 4, wherein each index in the target data table corresponds to two time fields, the two time fields include a time dimension type field and a time period field, and before the processing result generated by executing the second SQL file and aiming at the target index is written into the target data table, the method further comprises:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the modification of the two time fields corresponding to the target index is completed, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
6. A data table processing apparatus, the apparatus comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a processing request aiming at a target index in a target data table, and the processing request comprises a time dimension type and a time period;
the generating module is used for generating a time parameter according to the time dimension type and the time period;
the acquisition module is used for acquiring a first SQL file, the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL sentences, and codes used for representing time in the SQL sentences are expressed by adopting time variables;
the replacing module is used for replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and the writing module is used for writing the processing result which is generated by executing the second SQL file and aims at the target index into the target data table.
7. The apparatus of claim 6, further comprising
The segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
and the writing module is used for writing the corresponding processing result into the target database according to the type of each SQL statement after the time replacement.
8. An electronic device, comprising a processor and a memory, wherein at least one program code is stored in the memory, and wherein the at least one program code is loaded and executed by the processor to implement the data table processing method according to any one of claims 1 to 5.
9. A computer-readable storage medium, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the data table processing method according to any one of claims 1 to 5.
10. A computer program product, characterized in that the computer program product comprises computer program code, which is stored in a computer-readable storage medium, from which a processor of an electronic device reads the computer program code, the processor executing the computer program code, causing the electronic device to execute the data table processing method according to any of claims 1 to 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211045460.7A CN115470229B (en) | 2022-08-30 | 2022-08-30 | Data table processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211045460.7A CN115470229B (en) | 2022-08-30 | 2022-08-30 | Data table processing method and device, electronic equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115470229A true CN115470229A (en) | 2022-12-13 |
CN115470229B CN115470229B (en) | 2023-04-14 |
Family
ID=84368417
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211045460.7A Active CN115470229B (en) | 2022-08-30 | 2022-08-30 | Data table processing method and device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115470229B (en) |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103823680A (en) * | 2014-02-24 | 2014-05-28 | 广州精攻网络科技有限公司 | Development method and device for game service logic engine |
CN107908679A (en) * | 2017-10-26 | 2018-04-13 | 平安科技(深圳)有限公司 | Script sentence conversion method, device and computer-readable recording medium |
US20190005117A1 (en) * | 2017-06-30 | 2019-01-03 | Capital One Services, Llc | Systems and methods for code parsing and lineage detection |
CN109299132A (en) * | 2018-11-29 | 2019-02-01 | 中国人民财产保险股份有限公司 | SQL data processing method, system and electronic equipment |
CN111414377A (en) * | 2020-03-05 | 2020-07-14 | 微民保险代理有限公司 | Method and device for processing structured query statement and storage medium |
CN112380180A (en) * | 2020-11-17 | 2021-02-19 | 平安普惠企业管理有限公司 | Data synchronization processing method, device, equipment and storage medium |
CN113076333A (en) * | 2021-04-25 | 2021-07-06 | 中国建设银行股份有限公司 | Service abnormity monitoring method and device |
CN113391818A (en) * | 2021-03-26 | 2021-09-14 | 广州海量数据库技术有限公司 | Compiled file generation method and device |
CN113760864A (en) * | 2021-04-21 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Data model generation method and device |
CN113900944A (en) * | 2021-10-12 | 2022-01-07 | 北京沃东天骏信息技术有限公司 | Logic verification method and device applied to Flink SQL |
CN113918594A (en) * | 2021-09-27 | 2022-01-11 | 苏宁易购集团股份有限公司 | Method and device for generating structured query statement and computer equipment |
-
2022
- 2022-08-30 CN CN202211045460.7A patent/CN115470229B/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103823680A (en) * | 2014-02-24 | 2014-05-28 | 广州精攻网络科技有限公司 | Development method and device for game service logic engine |
US20190005117A1 (en) * | 2017-06-30 | 2019-01-03 | Capital One Services, Llc | Systems and methods for code parsing and lineage detection |
CN107908679A (en) * | 2017-10-26 | 2018-04-13 | 平安科技(深圳)有限公司 | Script sentence conversion method, device and computer-readable recording medium |
CN109299132A (en) * | 2018-11-29 | 2019-02-01 | 中国人民财产保险股份有限公司 | SQL data processing method, system and electronic equipment |
CN111414377A (en) * | 2020-03-05 | 2020-07-14 | 微民保险代理有限公司 | Method and device for processing structured query statement and storage medium |
CN112380180A (en) * | 2020-11-17 | 2021-02-19 | 平安普惠企业管理有限公司 | Data synchronization processing method, device, equipment and storage medium |
CN113391818A (en) * | 2021-03-26 | 2021-09-14 | 广州海量数据库技术有限公司 | Compiled file generation method and device |
CN113760864A (en) * | 2021-04-21 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Data model generation method and device |
CN113076333A (en) * | 2021-04-25 | 2021-07-06 | 中国建设银行股份有限公司 | Service abnormity monitoring method and device |
CN113918594A (en) * | 2021-09-27 | 2022-01-11 | 苏宁易购集团股份有限公司 | Method and device for generating structured query statement and computer equipment |
CN113900944A (en) * | 2021-10-12 | 2022-01-07 | 北京沃东天骏信息技术有限公司 | Logic verification method and device applied to Flink SQL |
Non-Patent Citations (2)
Title |
---|
THI-HAI-YEN VUONG;: "Learning to Transform Vietnamese Natural Language Queries into SQL Commands" * |
曾姣艳: "基于SQL模板的大数据批处理设计与实现" * |
Also Published As
Publication number | Publication date |
---|---|
CN115470229B (en) | 2023-04-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103810212A (en) | Automated database index creation method and system | |
CN110737594A (en) | Database standard conformance testing method and device for automatically generating test cases | |
CN110019116A (en) | Data traceability method, apparatus, data processing equipment and computer storage medium | |
CN110633258B (en) | Log insertion method, device, computer device and storage medium | |
CN114020774A (en) | Method, device and equipment for processing multiple rounds of question-answering sentences and storage medium | |
CN113468196A (en) | Method, apparatus, system, server and medium for processing data | |
CN110704635B (en) | Method and device for converting triplet data in knowledge graph | |
CN115470229B (en) | Data table processing method and device, electronic equipment and storage medium | |
CN117033309A (en) | Data conversion method and device, electronic equipment and readable storage medium | |
CN116661758A (en) | Method, device, electronic equipment and medium for optimizing log framework configuration | |
CN110727677B (en) | Method and device for tracing blood relationship of table in data warehouse | |
CN115757174A (en) | Database difference detection method and device | |
CN115310127A (en) | Data desensitization method and device | |
CN113419957B (en) | Rule-based big data offline batch processing performance capacity scanning method and device | |
CN109840213B (en) | Test data creating method, device, terminal and storage medium for GUI test | |
CN114496139A (en) | Quality control method, device, equipment and system of electronic medical record and readable medium | |
CN113609823A (en) | Method and device for editing questionnaire logic, terminal equipment and storage medium | |
CN111859985A (en) | AI customer service model testing method, device, electronic equipment and storage medium | |
CN112559331A (en) | Test method and device | |
CN112948485A (en) | Question-answer data synchronization method, device, system, server and storage medium | |
CN117874211B (en) | Intelligent question-answering method, system, medium and electronic equipment based on SAAS software | |
CN115292194B (en) | Method for debugging flow, electronic equipment and computer readable storage medium | |
CN112130841B (en) | SQL development method and device and terminal equipment | |
CN110309038B (en) | Performance test method and device, electronic equipment and computer readable storage medium | |
CN115981657A (en) | Code generation method and device, electronic equipment and readable medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |