CN117032661A - Real-time computing visualization development realization method and device and computer equipment - Google Patents
Real-time computing visualization development realization method and device and computer equipment Download PDFInfo
- Publication number
- CN117032661A CN117032661A CN202310685720.5A CN202310685720A CN117032661A CN 117032661 A CN117032661 A CN 117032661A CN 202310685720 A CN202310685720 A CN 202310685720A CN 117032661 A CN117032661 A CN 117032661A
- Authority
- CN
- China
- Prior art keywords
- visual
- processing logic
- real
- configuration
- semantic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011161 development Methods 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000012800 visualization Methods 0.000 title claims abstract description 56
- 230000000007 visual effect Effects 0.000 claims abstract description 130
- 238000012545 processing Methods 0.000 claims abstract description 103
- 238000004364 calculation method Methods 0.000 claims abstract description 23
- 238000012795 verification Methods 0.000 claims description 31
- 230000015654 memory Effects 0.000 claims description 26
- 238000004458 analytical method Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 14
- 230000003993 interaction Effects 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 13
- 238000010276 construction Methods 0.000 claims description 7
- 230000010354 integration Effects 0.000 claims description 5
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000010365 information processing Effects 0.000 claims 1
- 238000003672 processing method Methods 0.000 claims 1
- 230000002452 interceptive effect Effects 0.000 description 7
- 229910021532 Calcite Inorganic materials 0.000 description 5
- 239000008186 active pharmaceutical agent Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/42—Syntactic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of real-time computing, and discloses a method, a device and computer equipment for realizing visual development of real-time computing, wherein the method comprises the following steps: acquiring semantic rule information calculated in real time, wherein the semantic rule file is used for describing a grammar structure of a data processing logic file in a real-time calculation service; constructing data query statement information according to the semantic rule information, and generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not separable in the data query statement information; generating a plurality of visualization configuration items for a plurality of leaf nodes of the processing logic tree; based on the structure of the processing logic tree, connecting a plurality of visual configuration items to construct and generate a visual development topological graph. The invention can carry out fine granularity visual configuration, and greatly improves the real-time calculation function range covered by visual development.
Description
Technical Field
The invention relates to the technical field of real-time computing, in particular to a method and a device for realizing visual development of real-time computing and computer equipment.
Background
The real-time computing technology is applied to data processing scenes with higher real-time requirements, such as real-time ETL, real-time monitoring, real-time transaction inquiry and other services, and the requirements of the services on the time delay of the computing result reach millisecond level or even shorter. At present, the Flink is a mainstream real-time computing development framework, adopts a stream processing engine driven by an event, is a design idea of regarding batches as a limited stream, and has the characteristics of high throughput, low delay and high performance.
In the development of actual computing operation, developers, business personnel, flight big data analysts and the like are required to write the flight SQL for representing the data processing logic according to actual requirements, and finally the flight SQL is uploaded to a computing server or a data resource platform to execute real-time computation. The method has higher professional quality on the aspects of data processing, SQL, flink components, related APIs and the like for related staff, and brings difficulty to real-time computing application landing and application popularization.
In order to enable business personnel to get on hand more quickly, some related technologies provide a visual operation development scheme, developers develop visual interaction interfaces and provide real-time calculation development topological diagrams which are convenient to understand and operate in the interaction interfaces, and the business personnel utilize the topological diagrams to carry out visual configuration on data processing logic of real-time calculation. However, when developing the visual interactive interface, the developer mostly sets the configuration items according to the self working experience or certain specific service requirements set by the service personnel, so that the configuration granularity of the configuration items is coarse and the configuration granularity of a plurality of configuration items is not uniform, and the coverage of the functional range is not comprehensive enough.
Disclosure of Invention
In view of this, the embodiment of the invention provides a method, a device and a computer device for realizing the visual development of real-time computation, which can perform fine-granularity visual configuration and greatly improve the real-time computation function range covered by the visual development.
In a first aspect, an embodiment of the present invention provides a method for implementing real-time computing visualization development, where the method includes:
acquiring semantic rule information calculated in real time, wherein the semantic rule information file is used for describing a grammar structure of a data processing logic file in a real-time calculation service;
constructing data query statement information according to the semantic rule information, and generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not separable in the data query statement information;
generating a plurality of visualization configuration items for a plurality of leaf nodes of the processing logic tree;
based on the structure of the processing logic tree, connecting a plurality of visual configuration items to construct and generate a visual development topological graph.
In a second aspect, an embodiment of the present invention provides a real-time computing visualization development implementation apparatus, where the apparatus includes:
The grammar rule determining module is used for acquiring semantic rule information of real-time calculation, wherein the semantic rule information is used for describing the grammar structure of a data processing logic file in the real-time calculation service;
the logic tree construction module is used for constructing data query statement information according to the semantic rule information, generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not detachable in the data query statement information;
the configuration item generation module is used for generating a plurality of visual configuration items aiming at a plurality of leaf nodes of the processing logic tree;
and the topological graph construction module is used for connecting a plurality of visual configuration items based on the structure of the processing logic tree so as to construct and generate a visual development topological graph.
In a third aspect, an embodiment of the present invention provides a computer apparatus, including: the system comprises a memory and a processor, wherein the memory and the processor are in communication connection, the memory stores computer instructions, and the processor is used for enabling a computer to execute the real-time computing visualization development implementation method of the first aspect or any implementation mode corresponding to the first aspect by executing the computer instructions.
From the above, the method, the device and the computer equipment for implementing real-time computing visualization development provided by the embodiments of the present specification have the following beneficial technical effects:
the method comprises the steps of obtaining semantic rule information of real-time computation, constructing data query statement information based on the semantic rule information, carrying out semantic splitting on the data query statement information, thereby generating a processing logic tree capable of structurally representing real-time computation data processing logic, and generating a plurality of visual configuration items for a plurality of leaf nodes in the processing logic tree to construct the visual development topological graph. The leaf nodes in the processing logic tree correspond to basic semantic elements which cannot be split any more in the data query statement information, so that the configuration of the finest granularity of the specific data processing logic can be calculated in real time by utilizing a plurality of visual configuration items which are generated corresponding to a plurality of leaf nodes, and the functional range of the visual development of the real-time calculation, which can be covered by the plurality of visual configuration items, is more comprehensive.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow diagram of a method for implementing real-time computing visualization development in accordance with some embodiments of the invention;
FIG. 2 is a schematic diagram of semantic rule information in a real-time computing visualization development implementation method according to some embodiments of the present invention;
FIG. 3 is a flow chart of a method for generating a processing logic tree in a real-time computing visualization development implementation method according to some embodiments of the invention;
FIG. 4 is a schematic diagram of the structure of a processing logic tree in a real-time computing visualization development implementation method according to some embodiments of the invention;
FIG. 5 is a flow chart of a method for generating a string configuration item in a real-time computing visualization development implementation method according to some embodiments of the invention;
FIG. 6 is a flow chart of a method for splitting string configuration items in a real-time computing visualization development implementation method according to some embodiments of the invention;
FIG. 7 is a flow chart of a method for performing verification in a real-time computing visualization development implementation method according to some embodiments of the invention;
FIG. 8 is a block diagram of a real-time computing visualization development implementation apparatus in accordance with an embodiment of the present invention;
fig. 9 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The real-time computing technology is applied to data processing scenes with higher real-time requirements, such as real-time ETL, real-time monitoring, real-time transaction inquiry and other services, and the requirements of the services on the time delay of the computing result reach millisecond level or even shorter. At present, the Flink is a mainstream real-time computing development framework, adopts a stream processing engine driven by an event, is a design idea of regarding batches as a limited stream, and has the characteristics of high throughput, low delay and high performance.
In the development of actual computing operation, developers, business personnel, flight big data analysts and the like are required to write the flight SQL for representing the data processing logic according to actual requirements, and finally the flight SQL file is uploaded to a computing server or a data resource platform to execute real-time computation. The method has higher professional quality on the aspects of data processing, SQL, flink components, related APIs and the like for related staff, and brings difficulty to real-time computing application landing and application popularization.
In order to enable business personnel to get on hand more quickly, some related technologies provide a visual job development scheme, and developers provide visual development topological diagrams which are convenient to understand and operate, so that the business personnel can conveniently set and adjust specific data processing logic of real-time calculation by using the topological diagrams. However, when developing the visual interactive interface, the developer mostly sets the visual configuration items according to own working experience or certain specific service requirements set by service personnel, so that the configuration granularity is coarse and the configuration granularity of a plurality of configuration items is not uniform, and the coverage of the functional range is not comprehensive enough.
In view of the foregoing, an object of an embodiment of the present disclosure is to provide a method for implementing visual development of real-time computation, which generates data query statement information based on semantic rule information, analyzes the data query statement information to generate a corresponding processing logic tree, and generates a plurality of fine-grained visual configuration items for leaf nodes of the processing logic tree to construct a visual development topology map, so that service personnel can set and adjust specific data processing logic of real-time computation based on the visual development topology map.
Based on the above objects, in one aspect, an embodiment of the present specification provides a method for implementing real-time computing visualization development.
According to an embodiment of the present invention, there is provided a real-time computing visualization development implementation method embodiment, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different from that herein.
In this embodiment, a method for implementing real-time computing visualization development is provided, which may be used in the computer system described above, and fig. 1 is a flowchart of the method for implementing real-time computing visualization development according to an embodiment of the present invention, as shown in fig. 1, where the flowchart includes the following steps:
s101, semantic rule information of real-time calculation is obtained, and the semantic rule information is used for describing a grammar structure of a data processing logic file in a real-time calculation service.
The purpose of real-time computing development work is to generate real-time computing data processing logic files representing processing logic performed on corresponding data in a real-time computing business. Therefore, the grammar structure corresponding to the data processing logic file needs to be defined in real-time computing development. In this regard, real-time computed semantic rule information may be obtained that is used to express the grammar structure rules followed by the data processing logic file. The semantic rule information may be obtained using an input port of the computer system or directly from a specific database.
S102, constructing data query statement information according to the semantic rule information, and generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not separable in the data query statement information.
A real-time computing development component may be employed to construct the data query statement information according to the semantic rule information, where the data query statement information is an SQL (Structured Query Language ) -DQL (Data QueryLanguage, data query language) format file, and is used to describe a processing flow framework of the data flow when the target service performs real-time computing. The data query statement information includes a plurality of data query statements SQL-DQL.
The real-time computing development component can be selected from a Flink SQL API, a DataStream API, a Table API and the like. Accordingly, the grammar rule to be followed by the real-time computing service data processing logic file can be a flexible SQL BNF (Backus normal form, back us paradigm) grammar rule. The semantic rule information is used for describing the Flink SQL BNF grammar rule. The semantic rule information can be recorded in a text form, and the text of the semantic rule information describing the Flink SQL BNF grammar rule is shown in fig. 2.
The Flink SQL BNF can cover all the Flink SQL grammar, and construct the data query statement information according to the semantic rule information, so that the finally generated visual development topological graph can be ensured to accurately and comprehensively describe the real-time calculation data flow processing logic.
The data query statement information includes a plurality of data query statements SQL-DQL. In the Flink SQL BNF grammar rules, the data query statement SQL-DQL is also referred to as a query statement. The data query statement SQL-DQL/the query statement query takes the form of a select statement, i.e. "select field from table", meaning "select" field "from" table "to construct an SQL-DQL statement/query statement, so the SQL-DQL statement/query statement may also be referred to as a select statement.
In the data query term information, each query term may include a keyword skeleton, various types of items, expressions, and the like.
By carrying out semantic analysis on a plurality of query sentences in the data query sentence information, the query sentences can be split into a plurality of semantic nodes belonging to different logic levels, and the logic structure of the query sentences is determined.
For example, by means of semantic analysis, the data processing logic information can be first divided into a plurality of operator segments, which can be formed from one or more query statements, for implementing a complete data processing function. The query statement in the operator section can be further split into a plurality of statement nodes, and each statement node can be further split into a plurality of basic semantic elements, so that a logic structure corresponding to the data query statement information can be determined.
An example operator segment in the data processing logic information is as follows:
……
select*from source1 where id＝1
select count(*)from source2 group by id
select*from source1 inner join source2 on source1.id＝source2.id
……
in this operator section, 3 query statements are included, 3 of which are used to implement a complete data processing function. For the second query statement split, multiple basic semantic elements such as "count", "source", "group" and the like may be determined.
Corresponding to semantic nodes of each level in the data query statement information, the processing logic tree can be constructed and generated, and tree nodes of different levels in the processing logic tree correspond to semantic nodes of different logic levels in the data query statement information. For example, the processing logic tree includes nodes corresponding to operator segments, nodes corresponding to query sentences, and nodes corresponding to basic semantic elements. Nodes corresponding to the basic semantic elements which are not split any more in the data query statement information are leaf nodes in the processing logic tree.
S103, generating a plurality of visual configuration items aiming at a plurality of leaf nodes of the processing logic tree.
The basic semantic elements corresponding to the leaf nodes can be determined in the data query statement information, and the basic semantic elements and various associated optional configuration information and the like are packaged together to generate a visual configuration item. The visual configuration items may be displayed in a visual interactive interface for interactive operation configuration.
And the leaf nodes in the processing logic tree correspond to basic semantic elements, the basic semantic elements are minimum semantic nodes which are not split any more in the data query statement information, visual configuration items are generated aiming at the leaf nodes, and each visual configuration item can realize configuration operation on the minimum semantic nodes with the finest granularity in the data query statement information.
And S104, connecting a plurality of visual configuration items based on the structure of the processing logic tree to construct and generate a visual development topological graph. The visualization development topological graph is used for performing visualization configuration on data processing logic of real-time calculation.
And referring to the topological relation of a plurality of tree nodes in the processing logic tree, connecting a plurality of corresponding visual configuration items in the visual interaction interface so as to form the visual development topological graph. The visual development topological graph is used for the operation such as visual setting adjustment modification of specific data processing logic of real-time calculation by service personnel users, and further realizes the visual development of target service real-time calculation tasks.
According to the implementation method for the visual development of the real-time computation, semantic rule information of the real-time computation is obtained, data query statement information is constructed based on the semantic rule information, semantic splitting is conducted on the data query statement information, so that a processing logic tree capable of structurally representing processing logic of the real-time computation data is generated, and a plurality of visual configuration items are generated for a plurality of leaf nodes in the processing logic tree to construct the visual development topological graph. The leaf nodes in the processing logic tree correspond to basic semantic elements which cannot be split any more in the data query statement information, so that the configuration of the finest granularity of the specific data processing logic can be calculated in real time by utilizing a plurality of visual configuration items which are generated corresponding to a plurality of leaf nodes, and the functional range of the visual development of the real-time calculation, which can be covered by the plurality of visual configuration items, is more comprehensive.
As shown in fig. 3, in a real-time computing visualization development implementation method provided in one or more alternative embodiments of the present disclosure, generating a corresponding processing logic tree for the data query statement information includes:
s201, dividing the data query statement information into a plurality of operator segments, wherein the operator segments comprise at least one query statement.
The operator segment is a part of the data query statement information for realizing a complete service function, and can be formed by one or more query statements. And carrying out semantic analysis on the data query statement information based on the corresponding grammar rule, and dividing the data query statement information into a plurality of operator segments.
S202, respectively carrying out semantic splitting on a plurality of query sentences, and splitting the query sentences into a plurality of basic semantic elements.
The data query statement information includes a plurality of the query (query) statements, and a keyword skeleton, various items, expressions, and the like may be included in the query statement. And continuing to split a plurality of query sentences in the file by utilizing corresponding grammar rules, and determining a plurality of basic semantic elements forming the query sentences.
Elements such as keywords and item, expression can be included in each query sentence, and the elements can be further split into basic semantic elements including parameters, characters, connectors and the like through semantic splitting.
By carrying out semantic splitting on the data query statement information, three different logic-level semantic nodes from operator segments to statements to basic semantic elements can be determined, and corresponding processing logic trees can be constructed. In some alternative embodiments, the processing logic tree is generated by creating tree nodes of different levels for the semantic nodes of three different logic levels in the data query statement information.
S203, creating a plurality of operator nodes corresponding to the operator segments, and taking the operator nodes as child nodes of root nodes in the processing logic tree.
S204, establishing query nodes for the query sentences in the operator segments corresponding to the operator nodes, and taking the query nodes as child nodes of the operator nodes.
S205, establishing a semantic element node for the basic semantic element in the query statement corresponding to the query node, and taking the semantic element node as a child node of the query node.
As shown in fig. 4, an exemplary processing logic tree structure is schematically illustrated. As can be seen from the figure, the semantic element nodes corresponding to the basic semantic elements are leaf nodes in the processing logic tree.
In the real-time computing visualization development implementation method, a specific processing logic framework can be clearly carded out through semantic analysis on the data query statement information, a processing logic structure can be intuitively and clearly displayed by correspondingly generating the processing logic tree, and the leaf nodes in the processing logic tree correspond to the most basic semantic elements in the data query statement information. The basic semantic element is the minimum unit for modifying, configuring and adjusting data processing logic, and the processing logic tree is determined, so that a plurality of visual configuration items are conveniently created for the leaf nodes later, and service personnel can conveniently realize the finest granularity configuration by utilizing the visual configuration items.
In one or more embodiments of the present disclosure, a method for implementing real-time computing visualization development includes, for a plurality of leaf nodes of the processing logic tree, generating a plurality of visualization configuration items, including:
And packaging the basic semantic elements corresponding to the leaf nodes, and generating the visual configuration items to be displayed in a visual interaction interface.
Wherein, the basic semantic element itself and various associated optional configuration information and the like can be packaged together during packaging, thereby generating the visual configuration item. The visual configuration item can be displayed in a visual interaction interface and used for receiving and responding to a first configuration instruction, and the first configuration instruction is used for modifying and adjusting the basic semantic elements corresponding to the visual configuration item. The first configuration instruction may be an operation instruction of touch, clicking, selecting, dragging, editing and the like from a user, and is used for editing and configuring the corresponding visual configuration item of the leaf node on the finest granularity level.
In some optional real-time modes, when the first configuration instruction is received, the visual configuration item can provide a plurality of candidates, and further one of the candidates can be selected to replace the original basic semantic element according to the first configuration instruction. In this way, the business personnel user can flexibly modify and adjust the basic semantic elements corresponding to the visual configuration items.
In the practical service development process, the professional quality degree of different operators on aspects of data processing, SQL, flink components, related APIs and the like is considered to be different, and the development requirements in practice are different. For some business personnel, it may be necessary to configure the basic semantic elements involved in the data query statement information in fine granularity one by one. While other highly specialized staff may prefer coarse-grained configuration, it is desirable to directly modify the configuration of the entire or partial character string of a query statement in a document. Under the condition, the visualized configuration items with the finest granularity are still utilized to be configured one by one, so that the operation is too complicated for business personnel, the business personnel cannot flexibly switch between the coarse granularity and the fine granularity, and poor interactive operation experience of the business personnel can be caused. To address this problem, in some alternative embodiments of the present invention, fine-grained configuration items may be combined into coarse-grained string configuration items.
As shown in fig. 5, one implementation method for real-time computing visualization development provided in one or more alternative embodiments of the present disclosure further includes:
s401, receiving a second configuration instruction at the visual interaction interface, wherein the second configuration instruction is used for selecting a plurality of visual configuration items to be combined.
The service personnel user can input the second configuration instruction in the visual interaction interface, and the second configuration instruction can select a plurality of fine-granularity visual configuration items, so that the selected visual configuration items are combined and converted into coarse-granularity configuration items, and the service personnel can perform configuration more flexibly.
The second configuration instruction may be an operation instruction of touch control, clicking, selecting, dragging, editing and the like from a user, through which the user may select a plurality of fine-granularity visual configuration items in a mode of clicking one by one or selecting a plurality of common frames, and merge the fine-granularity visual configuration items into a larger-granularity visual configuration item, so that configuration on a coarse-granularity level may be performed.
S402, determining a plurality of visual configuration items selected by the second configuration instruction, and determining subtrees corresponding to the plurality of visual configuration items in the processing logic tree.
The visualization configuration items correspond to a plurality of leaf nodes in the processing logic book, and based on the plurality of leaf nodes and combined with the topological structure of the processing logic tree, the corresponding subtrees can be traced upwards to determine the corresponding query nodes or operator nodes of the plurality of leaf nodes.
S403, performing semantic integration on a plurality of nodes in the subtree, generating a character string corresponding to the subtree, and generating a character string configuration item for the character string.
And carrying out semantic integration on a plurality of nodes in the subtree, namely integrating semantic elements, query sentences or operators corresponding to the plurality of nodes. In some alternative embodiments, a calcite analysis engine may be employed for semantic integration. The calcite is an engine capable of converting any data query into an SQL-based query, and the calcite engine can realize functions of SQL analysis, verification, logic plan generation, logic plan optimization, physical plan generation and the like. The semantic elements corresponding to the nodes can be integrated into character strings conforming to grammar rules by using a calcite analysis engine.
After merging and integrating the plurality of fine-grained visual configuration items into the character string, generating the character string configuration item for the character string. The character string configuration item is used for receiving and responding to a third configuration instruction, and the third configuration instruction is used for modifying and adjusting the character string content corresponding to the character string configuration item. The third configuration instruction may be an operation instruction of touch control, clicking, selecting, dragging, editing and the like from a user, and through the third configuration instruction, the user can edit and modify specific character content in the corresponding character string configuration item.
In some optional real-time modes, when the third configuration instruction is received, the character string configuration item can provide an editable window, and a service personnel user can directly edit the character string by using the window so as to replace the character string corresponding to the character string configuration item. By the method, the business personnel user can flexibly modify and adjust the corresponding character string of the character string configuration item at the coarse granularity level.
In order to further optimize the interactive operation experience of business personnel, in some alternative embodiments of the present invention, the coarse-grained character string configuration items may be split into fine-grained visual configuration items.
As shown in fig. 6, one implementation method for real-time computing visualization development provided in one or more alternative embodiments of the present disclosure further includes:
s501, receiving a fourth configuration instruction at the visual interaction interface, wherein the fourth configuration instruction is used for splitting the character string configuration item.
The fourth configuration instruction may be an operation instruction of touch control, clicking, selecting, dragging, editing and the like from a user, through which the user can re-split the character string configuration item with a relatively coarse granularity into a plurality of visualized configuration items with a fine granularity, so as to adjust details of elements in the corresponding character string.
S502, splitting the character string corresponding to the character string configuration item into a plurality of basic semantic elements.
Similar to the semantic analysis of the data query statement information, the semantic analysis of the character string may be performed according to a corresponding grammar rule, and the character may be broken into a plurality of the basic semantic elements.
S503, generating a plurality of visual configuration items aiming at a plurality of basic semantic elements.
Similar to the basic semantic elements in the data query statement information, corresponding visual configuration items can be generated for the basic semantic elements in the character string.
In this way, business personnel users can flexibly decompose the coarse-grained level versus the string configuration items into a corresponding plurality of fine-grained visual configuration items.
In the real-time computing visual development implementation method, a service personnel user can flexibly switch between a fine-granularity visual configuration item and a coarse-granularity character string configuration item by using a second configuration instruction and a fourth configuration instruction, and the specific configuration granularity of the character string configuration item can be flexibly adjusted according to the specific content and the number of a plurality of visual configuration items selected by the second configuration instruction, so that different requirements of different service personnel users can be met to the greatest extent, and the user interactive operation experience is greatly optimized.
In some related technologies, since developers mostly set visual configuration items according to their own working experience or certain specific service requirements set by service personnel, the configuration granularity of the configuration items is coarse and is not uniform, and the configuration granularity level is fixed and cannot be flexibly adjusted. The granularity of verification of the configuration items also depends on the configuration granularity of the configuration items, and the coarser configuration granularity also leads to only coarser verification in the visual development process of the real-time computing operation, so that the practicability and stability of the whole service operation cannot be ensured. According to the technical scheme provided by some optional embodiments of the invention, fine granularity configuration can be realized, and the switching can be flexibly adjusted by configuring coarse granularity and fine granularity, so that fine granularity verification can be realized.
In one or more embodiments of the present disclosure, each operator segment in the data query statement information corresponds to a plurality of the visualization configuration items in the visualization development topology map. After generating a plurality of visual configuration items corresponding to one operator segment, the method further comprises checking the plurality of visual configuration items corresponding to the operator segment.
As shown in fig. 7, in some alternative embodiments, the method for verifying the plurality of visual configuration items corresponding to the operator may include the following steps:
s601, determining a statement segment corresponding to the operator in the data query statement information, and carrying out grammar analysis and verification on the statement segment.
The SQL sentence segments corresponding to the operators can be split into SqlNode code segments by adopting a calcite analysis engine, and analysis and verification based on grammar rules are carried out on the SqlNode code segments so as to determine whether the sentence segments corresponding to the operators accord with the grammar rules.
And S602, responding to the passing of grammar analysis verification, and performing metadata verification on the statement segment.
And extracting various metadata from the statement segment, and judging validity and correctness of the metadata. The metadata includes tables, fields, functions, and the like. Metadata verification includes determining whether a table, a field, a function in the statement section is valid, and determining whether the table, the field, and the type of the function and the corresponding method of use are correct.
S603, determining service information corresponding to the statement section in response to the passing of metadata verification, and performing physical rule verification on the service information.
The SqlNode code segment in the SQL sentence segment can be converted into a RelNode code, and the RelNode code is converted into a corresponding physical component in an actual service scene, so that service information in the actual service scene corresponding to the sentence segment is determined, and then physical rule verification can be performed on the service information. The physical rule checking refers to judging whether the service information meets some physical and service limiting conditions in the actual service application scene or not according to the service information.
And for the grammar analysis verification and the metadata verification, if the verification is wrong, prompting corresponding configuration items and SQL character strings can be aimed at so as to facilitate the inspection and maintenance of developers.
In some optional embodiments of the present disclosure, after the configuration of the portion corresponding to each operator in the visualization development topology map is completed, the configuration item of the portion is checked, and after the check is passed, the configuration item is saved. It can be understood by those skilled in the art that the verification can be performed immediately after each configuration item is generated, or uniformly after the entire visual development topology map is generated, and the specific time node selected by the verification can be flexibly set according to the actual requirement.
It should be noted that, in the data query statement information, the output of each operator may be used as the input of the next operator, and when the upstream operator is modified, the upstream operator check result may be transformed, and similarly, all the subsequent downstream operator check results may also be transformed. In this regard, in the implementation method for real-time computing visualization development provided in one or more alternative embodiments of the present disclosure, after modifying a configuration item in the visualization development topology map, the operator corresponding to the modified configuration item is determined, and verification is sequentially performed on the operator and a plurality of corresponding visualization configuration items of a downstream operator. And when the operators corresponding to the modified configuration items are checked for the operators and the downstream operators in sequence, the same checking method as the embodiment is adopted.
The method for realizing the visual development of the real-time computation checks a plurality of visual configuration items in the visual development topological graph corresponding to each operator, and performs complete check after each configuration item corresponding to the operator is modified and changed, so that the accuracy, the effectiveness and the stability of the visual development operation of the real-time computation can be effectively improved.
The embodiment also provides a device for realizing real-time computing visualization development, which is used for realizing the embodiment and the preferred implementation, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The embodiment provides a real-time computing visualization development implementation device, as shown in fig. 8, including:
the grammar rule determining module is used for acquiring semantic rule information of real-time calculation, wherein the semantic rule information is used for describing the grammar structure of a data processing logic file in the real-time calculation service;
the logic tree construction module is used for constructing data query statement information according to the semantic rule information, generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not detachable in the data query statement information;
the configuration item generation module is used for generating a plurality of visual configuration items aiming at a plurality of leaf nodes of the processing logic tree;
And the topological graph construction module is used for connecting a plurality of visual configuration items based on the structure of the processing logic tree so as to construct and generate a visual development topological graph.
In the real-time computing visualization development implementation apparatus provided in one or more optional embodiments of the present disclosure, the logic tree construction module is further configured to divide the data query statement information into a plurality of operators, where the operators include at least one query statement; respectively carrying out semantic splitting on a plurality of inquiry sentences, and splitting the inquiry sentences into a plurality of basic semantic elements; creating a plurality of operator nodes corresponding to a plurality of operators, and taking the operators as child nodes of root nodes in the processing logic tree; creating query nodes aiming at the query sentences in the operator nodes corresponding to the operator nodes, and taking the query nodes as child nodes of the operator nodes; and creating a semantic element node for the basic semantic element in the query statement corresponding to the query node, and taking the semantic element node as a child node of the query node.
In the real-time computing visual development implementation device provided by one or more optional embodiments of the present disclosure, the configuration item generating module is further configured to encapsulate the basic semantic element corresponding to the leaf node, and generate the visual configuration item for display in a visual interaction interface; the visual configuration item is used for receiving and responding to a first configuration instruction, and the first configuration instruction is used for modifying and adjusting the basic semantic elements corresponding to the visual configuration item.
The embodiment of the present disclosure provides a real-time computing visualization development implementation device, which further includes a configuration item merging module. The configuration item merging module is used for receiving a second configuration instruction at the visual interaction interface, and the second configuration instruction is used for selecting a plurality of visual configuration items to merge; determining a plurality of visual configuration items selected by the second configuration instruction, and determining subtrees corresponding to the plurality of visual configuration items in the processing logic tree; performing semantic integration on a plurality of nodes in the subtree to generate a character string corresponding to the subtree, and generating a character string configuration item aiming at the character string; the character string configuration item is used for receiving and responding to a third configuration instruction, and the third configuration instruction is used for modifying and adjusting the character string content corresponding to the character string configuration item.
The embodiment of the present disclosure provides a real-time computing visualization development implementation device, which further includes a configuration item disassembly module. The configuration item disassembling module is used for receiving a fourth configuration instruction at the visual interaction interface, and the fourth configuration instruction is used for disassembling the character string configuration item; splitting the character string corresponding to the character string configuration item into a plurality of basic semantic elements; for a plurality of the basic semantic elements, a plurality of visual configuration items are generated.
In one or more embodiments of the present disclosure, a real-time computing visualization development implementation apparatus is provided, where each operator in the data query statement information corresponds to a plurality of the visualization configuration items in the visualization development topology map. The device also comprises a configuration item checking module, wherein the configuration item checking module is used for checking a plurality of visual configuration items corresponding to one operator after generating the plurality of visual configuration items corresponding to the operator.
In the real-time computing visualization development implementation device provided by one or more optional embodiments of the present disclosure, the configuration item checking module is further configured to determine a statement segment corresponding to the operator in the data query statement information, and perform syntax parsing check on the statement segment; responding to the grammar analysis verification passing, and performing metadata verification on the statement segment; the metadata verification includes determining whether a table, a field, a function in the statement section are valid, and determining whether the table, the field, the type of the function and the corresponding usage method are correct; and responding to the passing of metadata verification, determining service information corresponding to the statement section, and carrying out physical rule verification on the service information.
In the implementation device for real-time computing visualization development provided in one or more optional embodiments of the present disclosure, after a configuration item in the visualization development topological graph is modified, the configuration item verification module is further configured to determine the operator corresponding to the modified configuration item, and verify the operator and a plurality of corresponding visualization configuration items of a downstream operator in sequence.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The embodiment of the invention also provides computer equipment, which is provided with the device for realizing the real-time computing visualization development shown in the figure 8.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 9, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 9.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created from the use of the computer device of the presentation of a sort of applet landing page, and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (10)
1. A method for implementing real-time computing visualization development, the method comprising:
acquiring semantic rule information calculated in real time, wherein the semantic rule information file is used for describing a grammar structure of a data processing logic file in a real-time calculation service;
constructing data query statement information according to the semantic rule information, and generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not separable in the data query statement information;
generating a plurality of visualization configuration items for a plurality of leaf nodes of the processing logic tree;
based on the structure of the processing logic tree, connecting a plurality of visual configuration items to construct and generate a visual development topological graph, wherein the visual development topological graph is used for carrying out visual configuration on data processing logic of real-time calculation.
2. The method of claim 1, wherein generating a corresponding processing logic tree for the data query statement information comprises:
dividing the data query statement information into a plurality of operators, wherein the operators comprise at least one query statement;
respectively carrying out semantic splitting on a plurality of inquiry sentences, and splitting the inquiry sentences into a plurality of basic semantic elements;
creating a plurality of operator nodes corresponding to a plurality of operators, and taking the operators as child nodes of root nodes in the processing logic tree;
creating query nodes aiming at the query sentences in the operator nodes corresponding to the operator nodes, and taking the query nodes as child nodes of the operator nodes;
and creating a semantic element node for the basic semantic element in the query statement corresponding to the query node, and taking the semantic element node as a child node of the query node.
3. The method of claim 1, wherein generating a plurality of visualization configuration items for a plurality of leaf nodes of the processing logic tree comprises:
packaging the basic semantic elements corresponding to the leaf nodes, and generating the visual configuration items to be displayed in a visual interaction interface;
The visual configuration item is used for receiving and responding to a first configuration instruction, and the first configuration instruction is used for modifying and adjusting the basic semantic elements corresponding to the visual configuration item.
4. The method according to claim 1, wherein the method further comprises:
receiving a second configuration instruction at the visual interaction interface, wherein the second configuration instruction is used for selecting a plurality of visual configuration items to be combined;
determining a plurality of visual configuration items selected by the second configuration instruction, and determining subtrees corresponding to the plurality of visual configuration items in the processing logic tree;
performing semantic integration on a plurality of nodes in the subtree to generate a character string corresponding to the subtree, and generating a character string configuration item aiming at the character string;
the character string configuration item is used for receiving and responding to a third configuration instruction, and the third configuration instruction is used for modifying and adjusting the character string content corresponding to the character string configuration item.
5. The method according to claim 4, wherein the method further comprises:
receiving a fourth configuration instruction at the visual interaction interface, wherein the fourth configuration instruction is used for splitting the character string configuration item;
Splitting the character string corresponding to the character string configuration item into a plurality of basic semantic elements;
for a plurality of the basic semantic elements, a plurality of visual configuration items are generated.
6. The method of claim 2, wherein each operator in the data query statement information corresponds to a plurality of the visualization configuration items in the visualization development topology map;
after generating a plurality of visual configuration items corresponding to one operator, the method further comprises checking the plurality of visual configuration items corresponding to the operator.
7. The method of claim 6, wherein verifying the operator's corresponding plurality of visualization configuration items comprises:
determining a statement section corresponding to the operator in the data query statement information, and carrying out grammar analysis and verification on the statement section;
responding to the grammar analysis verification passing, and performing metadata verification on the statement segment;
the metadata verification includes determining whether a table, a field, a function in the statement section are valid, and determining whether the table, the field, the type of the function and the corresponding usage method are correct;
And responding to the passing of metadata verification, determining service information corresponding to the statement section, and carrying out physical rule verification on the service information.
8. The method of claim 6, further comprising, after modifying the configuration item in the visualization development topology map:
and determining the operator corresponding to the modified configuration item, and checking a plurality of visual configuration items corresponding to the operator and the downstream operator in sequence.
9. A real-time computing visualization development implementation apparatus, the apparatus comprising:
the grammar rule determining module is used for acquiring semantic rule information of real-time calculation, wherein the semantic rule information is used for describing the grammar structure of a data processing logic file in the real-time calculation service;
the logic tree construction module is used for constructing data query statement information according to the semantic rule information, generating a corresponding processing logic tree aiming at the data query statement information, wherein leaf nodes in the processing logic tree correspond to basic semantic elements which are not detachable in the data query statement information;
the configuration item generation module is used for generating a plurality of visual configuration items aiming at a plurality of leaf nodes of the processing logic tree;
The topology diagram construction module is used for connecting a plurality of visual configuration items based on the structure of the processing logic tree so as to construct and generate a visual development topology diagram, and the visual development topology diagram is used for carrying out visual configuration on the data processing logic of real-time calculation.
10. A computer device, comprising:
a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the information processing method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310685720.5A CN117032661A (en) | 2023-06-09 | 2023-06-09 | Real-time computing visualization development realization method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310685720.5A CN117032661A (en) | 2023-06-09 | 2023-06-09 | Real-time computing visualization development realization method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117032661A true CN117032661A (en) | 2023-11-10 |
Family
ID=88640055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310685720.5A Pending CN117032661A (en) | 2023-06-09 | 2023-06-09 | Real-time computing visualization development realization method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117032661A (en) |
-
2023
- 2023-06-09 CN CN202310685720.5A patent/CN117032661A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10162611B2 (en) | Method and apparatus for business rule extraction | |
| US11561772B2 (en) | Low-code development platform | |
| US10042903B2 (en) | Automating extract, transform, and load job testing | |
| US10747958B2 (en) | Dependency graph based natural language processing | |
| US10078843B2 (en) | Systems and methods for analyzing consumer sentiment with social perspective insight | |
| CN110941467A (en) | Data processing method, device and system | |
| CN108984155B (en) | Data processing flow setting method and device | |
| US9940380B2 (en) | Automatic modeling of column and pivot table layout tabular data | |
| US11611627B2 (en) | Action flow fragment management | |
| DE112018004660T5 (en) | USE COMMENTS TO PROVIDE OPTIMIZATION | |
| US11615076B2 (en) | Monolith database to distributed database transformation | |
| US20140101097A1 (en) | Template based database analyzer | |
| CN113268227A (en) | Zero-code visualization software development platform and development method | |
| CN115934097A (en) | Method, device, storage medium and electronic device for generating executable statement | |
| CN108932225B (en) | Method and system for converting natural language requirements into semantic modeling language statements | |
| WO2017097125A1 (en) | Executive code generation method and device | |
| US20230401484A1 (en) | Data processing method and apparatus, electronic device, and storage medium | |
| CN117032661A (en) | Real-time computing visualization development realization method and device and computer equipment | |
| CN113506099A (en) | Configuration system, method, computer device and storage medium for reporting service | |
| CN115904382A (en) | Code development method, system, client, server, device and storage medium | |
| US20240152511A1 (en) | Transliteration of machine interpretable languages for enhanced compaction | |
| CN116010444B (en) | Low-code interactive graph query statement construction method | |
| Peltomaa | Elasticsearch-based data management proof of concept for continuous integration | |
| US20240160459A1 (en) | Continuous integration emulator | |
| CN111159203B (en) | Data association analysis method, platform, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |