CN116756169A - Method and computing device for interactively generating data report - Google Patents
Method and computing device for interactively generating data report Download PDFInfo
- Publication number
- CN116756169A CN116756169A CN202310631670.2A CN202310631670A CN116756169A CN 116756169 A CN116756169 A CN 116756169A CN 202310631670 A CN202310631670 A CN 202310631670A CN 116756169 A CN116756169 A CN 116756169A
- Authority
- CN
- China
- Prior art keywords
- sql
- user
- data
- language model
- natural language
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000013507 mapping Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 2
- 230000002452 interceptive effect Effects 0.000 description 9
- 238000011161 development Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 239000013065 commercial product Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a method and computing equipment for interactively generating a data report. A method of interactively generating a data report comprising: receiving a natural language question of a user about a data report; generating prompt information of a template in a preset format according to the natural language question; taking the prompt information as input, outputting an intermediate SQL sentence by using a large language model for SQL task tuning, wherein the large language model for SQL task tuning has learned the table structure information and the SQL sentence in the preset application field to generate a specification; mapping the intermediate SQL statement into an executable SQL statement; acquiring an SQL result by using the executable SQL statement; and displaying the SQL result to the user. According to the technical scheme provided by the embodiment of the application, the SQL requirement of the data report can be effectively supported.
Description
Technical Field
The application relates to the technical field of big data and neural networks, in particular to a method and computing equipment for interactively generating a data report.
Background
In large data platforms such as e-commerce, there is a large amount of data and a large amount of database tables. There are often a variety of data requirements for these massive data and a large number of database tables by a manager or staff, such as querying statistics of the volume of a certain class of goods, the number of users purchased, etc. Due to the complexity of the application and the continuous updating, a large amount of manpower is required to execute tasks such as assisting running data or developing reports, so that the efficiency is low, and due to the fact that many queries are temporary and personalized, the query requirements cannot be supported particularly when e-commerce platforms are promoted on a large scale.
For this reason, there is a need to develop a more efficient data reporting solution.
Disclosure of Invention
The application provides a method and computing equipment for interactively generating a data report, which can effectively support the query requirement of the data report.
According to an aspect of the present application, there is provided a method of interactively generating a data report, comprising:
receiving a natural language question of a user about a data report;
generating prompt information of a template in a preset format according to the natural language question;
taking the prompt information as input, outputting an intermediate SQL sentence by using a large language model for SQL task tuning, wherein the large language model for SQL task tuning has learned the table structure information and the SQL sentence in the preset application field to generate a specification;
mapping the intermediate SQL statement into an executable SQL statement;
acquiring an SQL result by using the executable SQL statement;
and displaying the SQL result to the user.
According to some embodiments, the receiving a natural language question about a data report from a user includes:
and judging the validity of the natural language question, and filtering invalid questions and/or questions containing sensitive words.
According to some embodiments, the predetermined format template comprises: instruction type, instruction name, and natural language questions.
According to some embodiments, generating the prompt information of the template with the predetermined format according to the natural language question comprises:
and generating prompt information of a template in a preset format by utilizing the pre-trained large language model.
According to some embodiments, generating the prompt information of the template with the predetermined format according to the natural language question further comprises:
obtaining a history question similar to the natural language question from a history question library through similarity calculation;
and adding prompt information and intermediate SQL sentences corresponding to the historical questioning to be part of prompt information generated according to the natural language questioning.
According to some embodiments, the method further comprises training the large language model in advance to obtain the large language model optimized for the SQL task, wherein the training the large language model in advance comprises:
generating a specification based on the table structure information of the preset application field and the SQL sentence, and preparing corpus of the preset format template as model training data, wherein the model training data comprises virtual table structure information consistent with the table structure information of the preset application field, and the virtual table structure information comprises virtual table names and virtual field names which play a role in occupying space;
and optimizing the large language model by utilizing the model training data, so that the large language model learns the table structure information of the preset application field and the SQL sentence generation specification, and the large language model aiming at the SQL task is obtained.
According to some embodiments, the training the large language model in advance further comprises:
and obtaining a mapping table through the model training data, wherein the mapping table comprises mapping relations between the table structure information of the preset application field and the virtual table structure information.
According to some embodiments, mapping the intermediate SQL statement into an executable SQL statement comprises:
and replacing the virtual table names and the virtual field names in the intermediate SQL statement with real table names and field names by using the mapping table.
According to some embodiments, before the SQL results are obtained using the executable SQL statement, the method further comprises:
judging whether the user has corresponding data authority;
and if the user does not have the corresponding data authority, terminating the execution of the method and returning the authority prompt information to the user.
According to some embodiments, obtaining the SQL result using the executable SQL statement comprises:
and executing the executable SQL statement by using an application interface provided by the big data platform.
According to some embodiments, presenting the SQL results to the user comprises:
returning the SQL result to the user in a form of a table; and/or
And returning the chart drawn according to the SQL result according to the user requirement to the user.
According to another aspect of the present application, there is provided a computing device comprising:
a processor; and
a memory storing a computer program which, when executed by the processor, implements the method of any of the above.
According to another aspect of the application there is provided a non-transitory computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor, implement a method as claimed in any preceding claim.
According to another aspect of the application there is provided a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, implement the method as claimed in any one of the preceding claims.
According to the embodiment, based on the interactive question-and-answer mode, the natural language question is converted into the prompt information, the prompt information is utilized to generate the intermediate SQL sentence through a large language model which is optimized for the SQL task, and the intermediate SQL sentence is mapped into the executable SQL sentence, so that the query accuracy can be improved. According to the interactive question-and-answer type data report scheme of the embodiment, on one hand, a report user can obtain a desired data report only through interactive question, and the method is not limited to inquiring an existing report; on the other hand, due to the flexibility and universality of the report scheme, a large amount of manpower development resources can be saved, the efficiency is high, and a large amount of temporary and personalized query requirements can be supported under the condition of sudden tasks. In addition, according to some embodiments, the large model learns the table structure information and SQL generation specifications of the preset application field through the training corpus, and the tuning convergence speed of the large language model aiming at the SQL task of the preset field is high.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the description of the embodiments will be briefly described below.
FIG. 1 illustrates a flow chart of a method of interactively generating a data report in accordance with an example embodiment.
FIG. 2 illustrates a flowchart of a method for pre-training a large language model to obtain a large language model optimized for a predetermined SQL task, according to an example embodiment.
FIG. 3 shows a graph drawn based on SQL results according to an example embodiment.
FIG. 4 illustrates a block diagram of a computing device in accordance with an exemplary embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operators have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another element. Accordingly, a first component discussed below could be termed a second component without departing from the teachings of the present inventive concept. As used herein, the term "and/or" includes any one of the associated listed items and all combinations of one or more.
The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of related data is required to comply with the relevant laws and regulations and standards of the relevant country and region, and is provided with corresponding operation entries for the user to select authorization or rejection.
Those skilled in the art will appreciate that the drawings are schematic representations of example embodiments and that the modules or flows in the drawings are not necessarily required to practice the application and therefore should not be taken to limit the scope of the application.
Before describing embodiments of the present application, some terms or concepts related to the embodiments of the present application are explained.
Large model: the large model is a deep learning model with huge network scale, and can effectively process a large amount of data to obtain accurate prediction results. Large models are characterized by large parametric models, typically on the order of billions or even billions. The performance (referred to as accuracy) of a model is typically closely related to the parameter size of the model. The larger the model parameter scale, the more learning ability the model is.
A large language model (Large Language Model: LLM), also known as a large language model, is an artificial intelligence model, intended to understand and generate human language. The large language model is trained on a large amount of text data and can perform a wide range of tasks including text summarization, translation, emotion analysis, and the like. Large language models are large in scale, contain billions or even trillions of parameters, and are capable of learning complex patterns in language data. Large language models are typically based on deep learning architecture.
chatGLM: chatGLM is an open-source, conversational large language model supporting chinese-english bilingual questions and answers, and is optimized for chinese. The model is based on the GLM (general language model: generic language model) architecture, currently with nearly one billion parameters. chatGLM uses the concept of autoregressive gap filling, and a transducer-based encoder implementation performs better on NLU and whether or not a task is generated.
SQL: SQL is a structured query language (Structured Query Language), a language used to manage relational database systems. It allows users to retrieve, insert, update, and delete data from the database.
Data report: a data report is a means for presenting data in the form of tables or charts, etc., and is generally used for presenting and comparing data. A tabular data report typically includes columns and rows, each cell containing a data point or summary of data points. The data report in chart form may include pie charts, histograms, graphs, and the like.
In large data platforms such as e-commerce, due to the complexity of applications and the continual upgrades and updates, a great deal of manpower is required to perform tasks such as assisting running data or developing reports, which is not only inefficient, but also because many queries are temporary and personalized, especially when e-commerce platforms are promoted on a large scale, the query requirements cannot be supported. With the development of large language models, the inventor tries to develop a chat question-answer type data product which can support various query scenes and finally generate a data report to return to display based on the large language models. However, the inventors have found that while large language models are theoretically possible, it is not possible to deliver core application data and various table fields to the large language model for training, because of not only the training efficiency but also the safety in use.
Therefore, the application provides a method for interactively generating a data report, which supports a user to generate an intermediate SQL sentence in an interactive question-and-answer mode, maps the intermediate SQL sentence into an executable SQL sentence, acquires an SQL result by using the executable SQL sentence and displays the SQL result to the user.
Exemplary embodiments of the present application are described below with reference to the accompanying drawings.
FIG. 1 illustrates a flow chart of a method of interactively generating a data report in accordance with an example embodiment.
Referring to fig. 1, the method of interactively generating a data report according to an example embodiment is based on a large model, and a user only needs to interact with the tuned trained large language model, without having to scroll through various reports. For example, if the user inputs a trend chart of the past week volume, the technical solution of the present application may return the data of the past week volume in the form of the trend chart.
As shown in fig. 1, at S101, a natural language question of a user about a data report is received. For example, according to an example embodiment, a user may ask "divalent gmv of merchandise in the near 3 day child dimension".
According to some embodiments, validity judgment can be further performed on natural language questions, and invalid questions and/or questions containing sensitive words can be filtered.
At S103, a prompt message of a template of a predetermined format is generated from the natural language question. For example, the pre-formatted templates include instruction type, instruction name, and natural language questions, asking "divalent gmv of merchandise in the near 3 day subsidiaries dimension" for the previous user, see the following hints:
prompt (Prompt): "construction # SQL Query type select:" construction #5 month 1 day to 5 month 3 day each day subsidiary, commodity dimension divalent gmv ".
In the prompt information, the instruction type (instruction) is SQL Query, the instruction name is select, and the natural language question is "5 month 1 day to 5 month 3 days every day subsidiary, commodity dimension divalent gmv". The instruction name may be insert, update, create, or the like, in addition to select. It can be seen that "near 3 days" in the question has been converted to "5 months 1 day to 5 months 3 days per day".
According to some embodiments, a pre-trained large language model may be utilized to generate natural language questions into hints for a pre-formatted template. However, the present application is not limited thereto, and the natural language question may be generated into the prompt information of the template in a predetermined format by using a small model or the like.
In S105, the prompt information is used as input, the intermediate SQL statement is output by using the large language model tuned for the SQL task, and the large language model tuned for the SQL task has learned the table structure information and the SQL statement of the predetermined application field to generate a specification.
According to the embodiment of the application, the table structure information of each database table can be learned by a model by providing training corpus in a preset field. Thus, when using large model analysis reasoning, the model can output the corresponding SQL statement (select sum (xx field) from xx table group by xx field) by direct natural language questioning (e.g. I want to query gmv information on xxx days). In other words, in the application, the large model learns the table structure information and SQL generation specifications of the preset application field through the training corpus. The table of an application domain is generally not more than tens of sheets, the table structure information of the application domain can be quickly learned through fine tuning training, the efficiency is not wasted, and the convergence speed is also fast.
According to some embodiments, when generating the prompt information of the template with the predetermined format according to the natural language question, the historical question similar to the natural language question can be obtained from the historical question library through similarity calculation, and then the prompt information and the intermediate SQL sentence corresponding to the historical question are added as a part of the prompt information generated according to the natural language question. By the method, the large model can know the relation between the possible questions and the SQL sentences, so that the generation accuracy of the intermediate SQL sentences is further improved.
For example, for the prompt message "structure#sql Query type select × query#5month 1 day to 5 month 3 day every day subsidiaries, commodity dimension divalent gmv" input above, the large language model tuned for the SQL task may output the following intermediate SQL statement:
select b. Subsidiary, a. Commercial product, a. Ds, sum (a. Divalent gmv) as divalent gmv
from table_item a
left join table_company b
on a. commodity = b. commodity
where ds > = '5 month 1 day' and ds < = '5 month 3 day'
group by subsidiary, commodity, ds.
At S107, the intermediate SQL statement is mapped into an executable SQL statement.
When preparing the corpus data set of the tuning large language model, a mapping table containing the mapping relation between the table structure information of the preset application field and the virtual table structure information can be obtained according to the table structure information in the corpus. The virtual table structure information is consistent with the table structure information of the preset application field, and comprises virtual table names and virtual field names which play a role in occupying positions. In this way, the mapping table may be utilized to replace virtual table names and virtual field names in the intermediate SQL statement with real table names and field names, thereby mapping the intermediate SQL statement to an executable SQL statement.
Through mapping of table names and field names, such as table_item- > mmc_item_table, the previous intermediate SQL statement can be converted into the following truly executable SQL statement:
select b.company,a.item_id,a.ds,sum(a.pos_price_2)as gmv
from mmc_item_table a
left join mmc_subcompany_table b
on a.item_id=b.item_id
where ds>=‘20230501’and ds<=‘20230503’
group by company,item_id,ds。
at S109, the SQL result is obtained using the executable SQL statement. According to some embodiments, executable SQL statements may be executed using an application interface provided by a large data platform.
According to some embodiments, it may also be determined whether the user has corresponding data rights before the SQL result is obtained using the executable SQL statement. If the user does not have the corresponding data authority, the execution of the method is terminated, and the authority prompt information is returned to the user, so that the data security is ensured.
For example, according to some embodiments, an ACL (access control list) may be used for authentication, with only the administrator adding rights to conduct data queries. In addition, the rights can be graded, and different data grades can correspond to different rights. For example, an administrator may query all data, financial staff may query only some GMV-related data, commodity operators may query only commodity sales data, etc.
At S111, the SQL results are presented to the user.
According to some embodiments, SQL results may be returned to the user in tabular form, as shown in the examples of the tables below.
Company (Corp) | Commodity numbering | Date of day | Divalent gmv |
A | 001 | 5 months 1 day | 10 |
A | 001 | 5 months and 2 days | 11 |
A | 001 | 5 months 3 days | 12 |
B | 003 | 5 months 1 day | 20 |
B | 003 | 5 months and 2 days | 23 |
B | 003 | 5 months 3 days | 24 |
According to other embodiments, charts drawn according to SQL results on demand by the user are also returned to the user (see FIG. 3).
According to the method for interactively generating the data report based on the large language model, the natural language question is converted into the prompt information based on the mode of user interaction question and answer, the prompt information is utilized to generate the intermediate SQL sentence through the large language model which is optimized for the SQL task, and the intermediate SQL sentence is mapped into the executable SQL sentence. And then, acquiring the SQL result by using the executable SQL statement, and displaying the SQL result to the user. Thus, a brand new interactive question-and-answer type data report scheme can be realized. On the one hand, the report user can obtain the desired data report only through interactive questioning, and the report user is not limited to the SQL existing report. On the other hand, due to the flexibility and universality of the report scheme, a large amount of manpower development resources can be saved, the efficiency is high, and a large amount of temporary and personalized query requirements can be supported under the condition of sudden tasks.
FIG. 2 illustrates a flowchart of a method for pre-training a large language model to obtain a large language model optimized for a predetermined SQL task, according to an example embodiment.
Referring to fig. 2, at S201, a specification is generated based on table structure information of a predetermined application field and an SQL statement, and a corpus of a predetermined format template is prepared as model training data. The model training data contains virtual table structure information consistent with table structure information of a predetermined application field, and the virtual table structure information includes virtual table names and virtual field names that play a role of occupation.
For example, the predetermined format template may be of the form:
“content”“summary”
the training data of an example pre-formatted template may be the following corpus:
{ "content": "construction # SQL Query # -type # selection # -Query #5 month 1 day to 5 month 3 day each day of subsidiaries, commodity dimension divalent gmv", "discussion": "select b. Subsidiaries, a. Commodity, a. Ds, sum (a. Divalent gmv) as divalent gmv from table_ item a left join table _company b. Commodity = b, commodity where ds > = '5 month 1 day' and ds < = '5 month 3 day' group by subsidiaries, commodity, ds; "}.
And S203, optimizing the large language model by using the model training data, so that the large language model learns the table structure information and SQL sentence generation specifications of the preset application field, and the large language model aiming at the SQL task is obtained.
According to an embodiment, the table structure information of each database table can be learned by the model by providing the training corpus in a predetermined area. Thus, when using large model analysis reasoning, the model can output the corresponding SQL statement (select sum (xx field) from xx table group by xx field) by direct natural language questioning (e.g. I want to query gmv information on xxx days). In other words, in the application, the large model learns the table structure information and SQL generating specification of the preset application field. The table of an application domain is generally not more than tens of sheets, the table structure information of the application domain can be quickly learned through fine tuning training, the efficiency is not wasted, and the convergence speed is also fast.
According to some embodiments, the large language model may be tuned by various tuning modes based on prompt information, so as to obtain the large language model for the SQL task in the predetermined domain, for example, tuning by means of prefixed tuning, p-tuning, pro tuning, p-tuning v2, and the like, which will not be described herein.
In S205, a mapping table is obtained through the model training data, where the mapping table includes a mapping relationship between table structure information of a predetermined application area and virtual table structure information.
According to some embodiments, the mapping table may be stored in oss (object store access) or a database such as mysql.
When preparing the corpus data set of the tuning large language model, a mapping table containing the mapping relation between the table structure information of the preset application field and the virtual table structure information can be obtained according to the table structure information in the corpus. The virtual table structure information is consistent with the table structure information of the preset application field, and comprises virtual table names and virtual field names which play a role in occupying positions. In this way, the mapping table may be utilized to replace virtual table names and virtual field names in the intermediate SQL statement with real table names and field names, thereby mapping the intermediate SQL statement to an executable SQL statement.
FIG. 4 illustrates a block diagram of a computing device according to an example embodiment of the application.
As shown in fig. 4, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include a bus 22, a network interface 16, and an I/O interface 18. The processor 12, memory 14, network interface 16, and I/O interface 18 may communicate with each other via a bus 22.
The processor 12 may include one or more general purpose CPUs (Central Processing Unit, processors), microprocessors, or application specific integrated circuits, etc. for executing relevant program instructions. According to some embodiments, computing device 30 may also include a high performance display adapter (GPU) 20 that accelerates processor 12.
Memory 14 may include machine-system-readable media in the form of volatile memory, such as Random Access Memory (RAM), read Only Memory (ROM), and/or cache memory. Memory 14 is used to store one or more programs including instructions as well as data. The processor 12 may read instructions stored in the memory 14 to perform the methods according to embodiments of the application described above.
Computing device 30 may also communicate with one or more networks through network interface 16. The network interface 16 may be a wireless network interface.
Bus 22 may be a bus including an address bus, a data bus, a control bus, etc. Bus 22 provides a path for exchanging information between the components.
It should be noted that, in the implementation, the computing device 30 may further include other components necessary to achieve normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
Those skilled in the art will readily appreciate from the disclosure of the exemplary embodiments that the present disclosure may be readily utilized as a basis for modifying or modifying other embodiments of the present disclosure.
According to some embodiments, a completely new interactive question-and-answer data reporting scheme is provided. On the one hand, the report user can obtain the desired data report only through interactive questioning, and the report user is not limited to inquiring the existing report. On the other hand, due to the flexibility and universality of the report scheme, a large amount of manpower development resources can be saved, the efficiency is high, and a large amount of temporary and personalized query requirements can be supported under the condition of sudden tasks.
According to some embodiments, based on the interactive question-and-answer mode, the natural language question is converted into prompt information, the prompt information is utilized to generate an intermediate SQL sentence through a large language model optimized for the SQL task, and the intermediate SQL sentence is mapped into an executable SQL sentence, so that the SQL execution accuracy can be improved.
According to some embodiments, the large model learns the table structure information and SQL generation specifications of the preset application field through training corpus, and tuning convergence speed of the large language model aiming at the SQL task of the preset field is high.
It should be noted that, in the specific implementation process, the above technical solution may further include other components necessary for implementing normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
It should be noted that, for the sake of simplicity of description, the foregoing embodiments are all described as a series of combinations of actions, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously according to the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
It will be clear to a person skilled in the art that the solution according to the application can be implemented by means of software and/or hardware. "Unit" and "module" in this specification refer to software and/or hardware capable of performing a specific function, either alone or in combination with other components, where the hardware may be, for example, a field programmable gate array, an integrated circuit, or the like.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the cloud gateway according to the embodiments of the present application.
The present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method. The computer readable storage medium may include, but is not limited to, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of media or device suitable for storing instructions and/or data.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the embodiments above.
The exemplary embodiments of the present application have been particularly shown and described above. It is to be understood that this application is not limited to the precise arrangements, instrumentalities and instrumentalities described herein; on the contrary, the application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (12)
1. A method for interactively generating a data report, comprising:
receiving a natural language question of a user about a data report;
generating prompt information of a template in a preset format according to the natural language question;
taking the prompt information as input, outputting an intermediate SQL sentence by using a large language model for SQL task tuning, wherein the large language model for SQL task tuning has learned the table structure information and the SQL sentence in the preset application field to generate a specification;
mapping the intermediate SQL statement into an executable SQL statement;
acquiring an SQL result by using the executable SQL statement;
and displaying the SQL result to the user.
2. The method of claim 1, wherein receiving a natural language question from a user about a data report comprises:
and judging the validity of the natural language question, and filtering invalid questions and/or questions containing sensitive words.
3. The method of claim 1, wherein the predetermined format template comprises: instruction type, instruction name, and natural language questions.
4. The method of claim 3, wherein generating a reminder for a pre-formatted template from the natural language question comprises:
and generating prompt information of a template in a preset format by utilizing the pre-trained large language model.
5. The method of claim 3, wherein generating a reminder for a pre-formatted template from the natural language question further comprises:
obtaining a history question similar to the natural language question from a history question library through similarity calculation;
and adding prompt information and intermediate SQL sentences corresponding to the historical questioning to be part of prompt information generated according to the natural language questioning.
6. The method of claim 1, further comprising pre-training the large language model to obtain a large language model optimized for the SQL task, the pre-training the large language model comprising:
generating a specification based on the table structure information of the preset application field and the SQL sentence, and preparing corpus of the preset format template as model training data, wherein the model training data comprises virtual table structure information consistent with the table structure information of the preset application field, and the virtual table structure information comprises virtual table names and virtual field names which play a role in occupying space;
and optimizing the large language model by utilizing the model training data, so that the large language model learns the table structure information of the preset application field and the SQL sentence generation specification, and the large language model aiming at the SQL task is obtained.
7. The method of claim 6, wherein the pre-training the large language model further comprises:
case number: 230301CI
And obtaining a mapping table through the model training data, wherein the mapping table comprises mapping relations between the table structure information of the preset application field and the virtual table structure information.
8. The method of claim 7, wherein mapping the intermediate SQL statement to an executable SQL statement comprises:
and replacing the virtual table names and the virtual field names in the intermediate SQL statement with real table names and field names by using the mapping table.
9. The method of claim 1, wherein prior to obtaining the SQL results using the executable SQL statement, the method further comprises:
judging whether the user has corresponding data authority;
and if the user does not have the corresponding data authority, terminating the execution of the method and returning the authority prompt information to the user.
10. The method of claim 1, wherein obtaining the SQL results using the executable SQL statement comprises:
and executing the executable SQL statement by using an application interface provided by the big data platform.
11. The method of claim 1, wherein presenting the SQL results to the user comprises:
returning the SQL result to the user in a form of a table; and/or
And returning the chart drawn according to the SQL result according to the user requirement to the user.
12. A computing device, comprising:
a processor; and
memory storing a computer program which, when executed by the processor, implements the method according to any of claims 1-11.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310631670.2A CN116756169A (en) | 2023-05-30 | 2023-05-30 | Method and computing device for interactively generating data report |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310631670.2A CN116756169A (en) | 2023-05-30 | 2023-05-30 | Method and computing device for interactively generating data report |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116756169A true CN116756169A (en) | 2023-09-15 |
Family
ID=87959936
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310631670.2A Pending CN116756169A (en) | 2023-05-30 | 2023-05-30 | Method and computing device for interactively generating data report |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116756169A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117251473A (en) * | 2023-11-20 | 2023-12-19 | 摩斯智联科技有限公司 | Vehicle data query analysis method, system, device and storage medium |
CN117271561A (en) * | 2023-11-20 | 2023-12-22 | 海信集团控股股份有限公司 | SQL sentence generation method, device and equipment based on large language model |
CN117312372A (en) * | 2023-09-27 | 2023-12-29 | 星环信息科技(上海)股份有限公司 | SQL generating method, device, equipment and medium based on background knowledge enhancement |
CN117453717A (en) * | 2023-11-06 | 2024-01-26 | 星环信息科技(上海)股份有限公司 | Data query statement generation method, device, equipment and storage medium |
-
2023
- 2023-05-30 CN CN202310631670.2A patent/CN116756169A/en active Pending
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117312372A (en) * | 2023-09-27 | 2023-12-29 | 星环信息科技(上海)股份有限公司 | SQL generating method, device, equipment and medium based on background knowledge enhancement |
CN117312372B (en) * | 2023-09-27 | 2024-05-31 | 星环信息科技(上海)股份有限公司 | SQL generating method, device, equipment and medium based on background knowledge enhancement |
CN117453717A (en) * | 2023-11-06 | 2024-01-26 | 星环信息科技(上海)股份有限公司 | Data query statement generation method, device, equipment and storage medium |
CN117453717B (en) * | 2023-11-06 | 2024-07-09 | 星环信息科技(上海)股份有限公司 | Data query statement generation method, device, equipment and storage medium |
CN117251473A (en) * | 2023-11-20 | 2023-12-19 | 摩斯智联科技有限公司 | Vehicle data query analysis method, system, device and storage medium |
CN117271561A (en) * | 2023-11-20 | 2023-12-22 | 海信集团控股股份有限公司 | SQL sentence generation method, device and equipment based on large language model |
CN117271561B (en) * | 2023-11-20 | 2024-03-01 | 海信集团控股股份有限公司 | SQL sentence generation method, device and equipment based on large language model |
CN117251473B (en) * | 2023-11-20 | 2024-03-15 | 摩斯智联科技有限公司 | Vehicle data query analysis method, system, device and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11977568B2 (en) | Building dialogue structure by using communicative discourse trees | |
US11782985B2 (en) | Constructing imaginary discourse trees to improve answering convergent questions | |
CN116756169A (en) | Method and computing device for interactively generating data report | |
CN111670435B (en) | Using an exchange utterance tree to detect requests for interpretation | |
US11797593B2 (en) | Mapping of topics within a domain based on terms associated with the topics | |
JP2019504428A (en) | Web interface generation and test system based on machine learning | |
US20160162794A1 (en) | Decision tree data structures generated to determine metrics for child nodes | |
US20220230189A1 (en) | Discovery of new business openings using web content analysis | |
Bates et al. | Handling correlations between covariates and random slopes in multilevel models | |
Silva et al. | Integrating big data into the computing curricula | |
CN116541411A (en) | SQL sentence acquisition method, report generation device, computer equipment and storage medium | |
Lin et al. | Method of personalized educational resource recommendation based on LDA and learner’s behavior | |
CN113420542B (en) | Dialogue generation method, device, electronic equipment and storage medium | |
Newson et al. | Robit regression in Stata | |
US11803796B2 (en) | System, method, electronic device, and storage medium for identifying risk event based on social information | |
CN110428342B (en) | Data restoration method, server, customer service side and storage medium | |
US8880562B2 (en) | Generating a supplemental description of an entity | |
Soriano | Maximizing benefits from IT project management: from requirements to value delivery | |
CN116975254A (en) | Visual data presentation method, visual data presentation device and storage medium | |
US20230316340A1 (en) | Comic advertisement creation assistance system and comic advertisement creation assistance method | |
WO2022173397A1 (en) | A recommendation system using artificial intelligence algorithms | |
Zhao | Leveraging Data Mining Technique to Enhancing Online Education and Its Efficiency Study | |
CN111859985A (en) | AI customer service model testing method, device, electronic equipment and storage medium | |
CN113468280B (en) | Data cognition method, device, equipment and storage medium | |
CN118503396B (en) | Open prompt word-based ERP system large model calling method, device and medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |