WO2021008570A1 - 一种基于计算引擎的界面处理方法及装置 - Google Patents

一种基于计算引擎的界面处理方法及装置 Download PDF

Info

Publication number
WO2021008570A1
WO2021008570A1 PCT/CN2020/102226 CN2020102226W WO2021008570A1 WO 2021008570 A1 WO2021008570 A1 WO 2021008570A1 CN 2020102226 W CN2020102226 W CN 2020102226W WO 2021008570 A1 WO2021008570 A1 WO 2021008570A1
Authority
WO
WIPO (PCT)
Prior art keywords
result set
format
task
calculation engine
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/102226
Other languages
English (en)
French (fr)
Inventor
黄山
尹强
王和平
刘有
杨峙岳
邸帅
卢道和
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Publication of WO2021008570A1 publication Critical patent/WO2021008570A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24539—Query rewriting; Transformation using cached or materialised query results
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/248—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the embodiments of the present invention relate to the field of financial technology (Fintech), and in particular to an interface processing method and device based on a computing engine.
  • Fetech financial technology
  • Livy is an open source REST service based on Spark. It can submit tasks to Spark through code snippets, obtain the result set of task execution synchronously or asynchronously, and manage Spark contex. This simplifies the interaction between Spark and some applications, and realizes that Spark is used in some web applications.
  • Livy returns the result set of task execution directly to the operation interface, which is poor in readability and is not conducive to subsequent development by staff.
  • the embodiment of the present invention provides an interface processing method and device based on a calculation engine, which are used to improve the readability of the result set and facilitate subsequent development by the operator.
  • the embodiment of the present invention provides an interface processing method based on a calculation engine, including:
  • the result set identifier in the display request determine the result set in the first format for processing by the calculation engine; the result set is generated when each calculation engine processes historical tasks;
  • the metadata model is sent to the client, so that the client is displayed in a visual interface.
  • the result set generated by each calculation engine processing historical task is stored in the database, and the task executor can determine the result set of the corresponding format according to the result set identifier in the display request after receiving the display request sent by the client. And submit the first task corresponding to the result set to the calculation engine.
  • the calculation engine generates a temporary view based on the result set and returns the metadata of the temporary view.
  • the task executor then generates a metadata model based on the metadata of the temporary view, and reports it to the client
  • the terminal is displayed in a visual interface, and the data is highly readable, and the staff can perform the next operation according to the displayed interface, which is helpful for the staff to perform the next operation based on the result set data of the previous step.
  • storing the result set generated by each calculation engine processing historical tasks in the database is conducive to the storage of intermediate data or result sets.
  • the next task is submitted, there is no need to submit the task including the previous operation, and it can be directly from the database. Call the result set data to improve the operating efficiency of the calculation engine.
  • the method further includes:
  • the processing request includes metadata to be operated and operation instructions
  • the processing result returned by the calculation engine is received, and the processing result is added to the metadata model and then sent to the client.
  • the task executor can not only visually display the data of the result set previously stored in the database on the client, but also generate a second task for the result set and submit the second task to the calculation engine for execution. After the next operation, the result set generated after the operation is displayed on the client interface.
  • the staff can combine the visual interface of the client to carry out development operations. The result set data is readable and the development efficiency of the staff is improved.
  • the determining, according to the result set identifier in the display request, the result set in the first format for processing by the calculation engine includes:
  • the result set in the second format is a result set that is generated when each calculation engine processes historical tasks and is converted into a unified format
  • the result set in the second format is converted into the result set in the first format.
  • the conversion of the result set generated when the calculation engine processes historical tasks into a result set in a unified format includes:
  • the result set of the third format is converted into the result set of the second format.
  • the result set generated by each calculation engine when processing historical tasks is converted into a result set in a unified format and stored in the database, so that when other calculation engines are used to operate on the result set stored in the database, the unified Format the result set and convert the unified format result set into the format of the corresponding calculation engine.
  • an embodiment of the present invention also provides an interface processing device based on a calculation engine, including:
  • the transceiver unit is used to receive the display request sent by the client;
  • the processing unit is configured to determine, according to the result set identifier in the display request, a result set in the first format for processing by the calculation engine; the result set is generated when each calculation engine processes historical tasks;
  • the processing unit is further configured to generate a first task recording the result set in the first format, and submit the first task to the calculation engine;
  • the transceiver unit is further configured to receive metadata corresponding to the temporary view fed back by the computing engine and construct a visual metadata model; and send the metadata model to the client, so that the client can Visual interface display.
  • processing unit is further configured to:
  • the transceiver unit After sending the metadata model to the client, controlling the transceiver unit to receive a processing request submitted by the user on the visual interface; the processing request includes metadata to be operated and operation instructions;
  • Control the transceiver unit to receive the processing result returned by the calculation engine, add the processing result to the metadata model and send it to the client.
  • processing unit is specifically configured to:
  • the result set in the second format is a result set that is generated when each calculation engine processes historical tasks and is converted into a unified format
  • the result set in the second format is converted into the result set in the first format.
  • processing unit is specifically configured to:
  • the result set of the third format is converted into the result set of the second format.
  • an embodiment of the present invention also provides a computing device, including:
  • processor memory and communication interface; among them, the processor, memory and communication interface are connected by a bus;
  • the processor is configured to read the program in the memory, and execute the foregoing interface processing method based on the calculation engine;
  • the memory is used to store one or more executable programs, and can store data used by the processor when performing operations.
  • the embodiment of the present invention also provides a non-transitory computer-readable storage medium.
  • the non-transitory computer-readable storage medium stores computer instructions. When it runs on a computer, the computer executes the aforementioned calculation engine-based Interface processing method.
  • an embodiment of the present invention also provides a computer program product containing instructions.
  • the computer program product includes a calculation program stored on a non-transitory computer-readable storage medium.
  • the computer program includes program instructions. When the program instructions are executed by the computer, the computer executes the above-mentioned interface processing method based on the calculation engine.
  • FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of an interface processing method provided by an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another interface processing method according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an interface processing device provided by an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a computing device provided by this application.
  • FIG. 1 exemplarily shows a system architecture applicable to a computing engine-based interface processing method provided by an embodiment of the present invention.
  • the system architecture may include a client 100, an execution controller 200, a task executor 300, and a computing engine 400.
  • the client terminal 100 as a terminal for the user to write a development language, provides basic interactive interfaces such as editing, execution, and termination, and provides the user with a visual chart.
  • the execution controller 200 as a server, directly interacts with the client 100, is responsible for receiving codes and instructions submitted by users, maintaining the user's context status and task executor status information, and processing user requests according to the current status, so that the tasks submitted by the user Get timely implementation or provide users with relevant feedback information.
  • the task executor 300 is used to submit the task of the execution controller 200 to the calculation engine 400 and feed back the execution result of the calculation engine 400.
  • the computing engine 400 may be Spark, Hive, Hbase, etc.
  • FIG. 2 exemplarily shows the flow of a computing engine-based interface processing method provided by an embodiment of the present invention.
  • the flow can be executed by a computing engine-based interface processing device, which can be located in a task executor. In, it can be the task executor.
  • the process specifically includes:
  • Step 201 Receive a display request sent by the client.
  • Step 202 According to the result set identifier in the display request, determine the result set in the first format for processing by the calculation engine.
  • Step 203 Generate a first task recording the result set in the first format, and submit the first task to the calculation engine.
  • Step 204 Receive metadata corresponding to the temporary view fed back by the computing engine and construct a visual metadata model.
  • Step 205 Send the metadata model to the client, so that the client can display it in a visual interface.
  • the result set is generated when each calculation engine processes the historical task. Specifically, the task executor submits the task to the calculation engine, and the calculation engine returns the execution result, which is generated when the calculation engine processes the historical task. Further, the execution result can be understood as a result set with a specific format.
  • the format of the result set is determined by the type of calculation engine. For example, if the calculation engine is Spark, the result set format returned by the Spark engine can be processed by the Spark engine. Data Format.
  • Each calculation engine generates a result set in a specific format when processing historical tasks, and returns the result set to the task executor.
  • each calculation is recorded in the task executor
  • the formatting class corresponding to the result set of the engine.
  • the task executor After the task executor receives the result set generated by the calculation engine when processing historical tasks, it will determine the formatting class of the result set according to the result set, and according to the formatting of the result set Class to convert the result set into a unified format.
  • the task executor is connected to the database, and after the result set is converted into a unified format, the result set with the unified format can be stored in the database.
  • the database can be a distributed database, or HDFS (Hadoop Distributed File System, distributed file system).
  • the result set generated by each calculation engine when processing historical tasks is equivalent to the result set in the third format, and the result set in the unified format stored in the database is equivalent to the result set in the second format.
  • the task executor receives the result set of the third format generated when the calculation engine processes the historical task; determines the formatting class of the result set of the third format according to the result set of the third format; The formatting class of the result set of the third format, which parses the result set of the third format into the result set of the second format.
  • a result set formatting class corresponding to all development languages is recorded in the task executor.
  • the task executor finds the formatting class corresponding to the result set of the language according to the development language type submitted by the user, and parses the original result set according to the formatting class, and determines the metadata information and metadata in the original result set The value of, where metadata information includes fields, field types, sorting methods, etc.
  • the task executor converts the parsed metadata information into a preset format, such as converting the python numbe in the result set into a double.
  • the result set converted into a preset format is stored in the database for subsequent data reading.
  • the task executor receives the display request sent by the client, and determines the result set in the first format for processing by the calculation engine according to the result set identifier in the display request.
  • the result set identifier is recorded in the display request, and the task executor can determine the storage path of the result set in the second format according to the result set identifier in the display request.
  • the second format is the result set stored in the database. Unified format.
  • the task executor determines the result set of the second format from the database according to the storage path of the result set of the second format, and converts the result set of the second format into the result of the first format according to the identification of the calculation engine in the display request set. In other words, after the task executor reads the result set in a unified format, it will convert the result set into a result set in the format used by the calculation engine.
  • the task executor receives a display request sent by the client.
  • the display request is used to instruct to generate a task based on the result set and submit it to Spark for execution.
  • the task executor determines the result set in the second format, it will After the result set of the second format is converted into the result set of Spark format, a Spark task is generated and submitted.
  • the task executor can read the result set in the second format from HDFS according to the storage path of the result set in the second format, and display it as a ResultSetReader object in the memory. This object holds all metadata information and metadata values of the result set in the second format.
  • the task executor converts the metadata information (field name and field type) of the result set in the second format into a metadata type (StructType) supported by Spark.
  • the field type is converted by querying a one-to-one correspondence, and each field type in the second format has a corresponding Spark DataType.
  • the field name directly follows the field name in the second format.
  • the task executor converts the value of the metadata in the second format into the value type of Spark (Row).
  • each row of values is an array, and the constructor of Spark Row accepts an array of values as Parameters, you can directly traverse all rows of the result set object, pass the array to the Row construction method, and obtain a series of related Row objects.
  • the obtained StructType and Row objects are the result set in Spark format.
  • the task executor converts the result set generated by each calculation engine when processing historical tasks into a unified format result set and stores it in the database, which is convenient for reading when other calculation engines are used to operate the result set stored in the database. Take the result set of the unified format and convert the result set of the unified format into the format of the corresponding calculation engine. For example, the task executor stores the result set in the Spark format in the database in a unified format, and then can read the result set in the unified format and convert it into a result set in the Hive format for calculation in Hive. In this way, tasks can be submitted to execution environments such as Spark, Hive, and Shell.
  • the task executor can generate the first task that records the result set of the first format, and submit the first task to the calculation Engine, the calculation engine creates a temporary view in the calculation engine according to the first task, and feeds back the metadata in the temporary view to the task executor, and the task executor builds a visual metadata model based on the metadata, and visualizes the built
  • the metadata model is sent to the client, and the client displays the visual interface.
  • the metadata in the temporary view can be understood as the above-mentioned metadata types, including field types, field names, and so on.
  • the computing engine may be a Spark engine.
  • the task executor determines the result set in the Spark format, it can first determine whether there is a Spark context corresponding to the first task in the current Spark engine. If not, start one Spark context, if available, directly pass the result set in Spark format to Spark’s createDataFrame method. After obtaining a Dataframe, call the createOrReplaceTempView method provided by Spark to submit the code snippet for creating the temporary view to Spark for execution. Create a temporary view in Spark context.
  • the Spark engine feeds back the metadata in the temporary view to the task executor, and the task executor creates a metadata model, which can be stored in the json format.
  • the metadata model contains all the fields and field type information (column and datatype arrays), and can be used to carry information such as indicators, dimensions, and filter conditions defined later (also implemented through an array of objects) .
  • the metadata model will be returned to the client for analysis in the form of json, and the client can display all the fields owned by the metadata model.
  • the client displays the metadata model.
  • the user can perform arithmetic operations on the display interface of the client, the user can submit processing requests on the visual interface, and the client submits the data submitted by the user.
  • the processing request is sent to the task executor.
  • the processing request may include the metadata to be operated and the operation instruction.
  • the task executor determines the temporary view corresponding to the metadata to be operated from the context corresponding to the first task according to the processing request, and Process the request and the temporary view corresponding to the metadata to be operated, generate a second task, and submit the second task to the calculation engine; the calculation engine performs calculation operations according to the second task.
  • the calculation engine determines the metadata correspondence from the temporary view
  • the value corresponding to the data is executed by the above operation instruction to obtain the processing result, that is, the result set corresponding to the second task is obtained.
  • the task executor receives the processing result returned by the calculation engine, adds the processing result to the metadata model, and sends it to the client.
  • the client interface can provide a visualization button.
  • the task executor After the task executor returns the result set of the calculation engine, the user can determine whether the result set meets the user's needs through the client interface, and after confirming that it meets the needs, The result set is submitted to the task executor through the visualization button. Further, the task executor performs the above operations and displays the metadata model corresponding to the result set on the client interface.
  • the client interface can display various fields according to the metadata model , And compose information such as indicators, dimensions, and filter conditions. Users can choose the calculation method for metadata on the client. For example, if you want to aggregate a field, drag it into the indicator area and select the method of aggregation calculation Save later.
  • the task executor can be spliced into SQL (Structured Query Language, structured query language) code fragments according to metadata and operation methods. For example, if the A field is summed, it is converted into a sum(A) code fragment.
  • the task executor submits the SQL code fragments to the calculation engine.
  • the calculation engine generates processing results based on the SQL code fragments.
  • the task executor fills the processing results into the metadata model and sends them to the client.
  • the client fills in the data according to the chart mode selected by the user After entering the chart, rendering is performed for the final visual display.
  • a visualization service module may be set, and the visualization service module may be set in the above-mentioned task executor to complete part of the tasks of the task executor.
  • the visual service module can also be set in the execution controller or an independent server, which is not limited here.
  • another visualization flowchart as shown in FIG. 3 can be used, and the client, task executor, and calculation engine are also involved in the process.
  • Step 301 the client generates a visualization request
  • Step 302 The client sends the visualization request to the visualization service module
  • Step 303 The visualization service module creates a temporary view task according to the visualization request
  • Step 304 The visualization service module submits the temporary view task to the calculation engine via the task executor;
  • Step 305 the calculation engine creates a temporary view
  • Step 306 The visualization service module sends the temporary view query request to the task executor;
  • Step 307 The task executor queries the metadata of the temporary view from the calculation engine
  • Step 308 The task executor generates a metadata model based on the metadata of the temporary view
  • Step 309 The task executor sends the metadata model to the client through the visualization module
  • Step 310 the client displays the draggable metadata front-end component
  • Step 311 The client interface sends configuration information such as indicators and dimensions to the visualization service module;
  • Step 312 the visualization service module splices SQL statements to generate a query task
  • Step 313 The visualization service module sends the query task to the calculation engine via the task executor;
  • Step 314 The calculation engine generates a query result set according to the query task
  • Step 315 The calculation engine sends the query result set to the task executor
  • Step 316 The task executor formats the query result and generates a result set in a unified format
  • Step 317 The task executor fills the unified format result set into the metadata model as the data to be displayed;
  • Step 319 The task executor sends the data to be displayed to the client interface via the visualization service module;
  • step 320 the client interface is rendered into a chart.
  • the task executor stores the result sets generated by each calculation engine processing historical tasks in the database.
  • the task executor can determine the corresponding format according to the result set identifier in the display request after receiving the display request sent by the client And submit the first task corresponding to the result set to the calculation engine.
  • the calculation engine generates a temporary view based on the result set and returns the metadata of the temporary view.
  • the task executor then generates a metadata model based on the metadata of the temporary view , And displayed on the client as a visual interface, the data is highly readable, and the staff can perform the next operation according to the displayed interface, which is helpful for the staff to perform the next operation based on the result set data of the previous step.
  • storing the result set generated by each calculation engine processing historical tasks in the database is conducive to the storage of intermediate data or result sets.
  • the next task is submitted, there is no need to submit the task including the previous operation, and it can be directly from the database. Call the result set data to improve the operating efficiency of the calculation engine.
  • FIG. 4 exemplarily shows the structure of an interface processing device based on a computing engine provided by an embodiment of the present invention, and the device can execute the flow of the interface processing method based on the computing engine.
  • the device includes:
  • the transceiver unit 401 is configured to receive a display request sent by the client;
  • the processing unit 402 is configured to determine, according to the result set identifier in the display request, a result set in the first format used for processing by the calculation engine; the result set is generated when each calculation engine processes historical tasks;
  • the processing unit 402 is further configured to generate a first task recording the result set in the first format, and submit the first task to the calculation engine;
  • the transceiving unit 401 is further configured to receive metadata corresponding to the temporary view fed back by the calculation engine and construct a visual metadata model; and send the metadata model to the client, so that the client Displayed in a visual interface.
  • processing unit 402 is further configured to:
  • control the transceiver unit 401 After the metadata model is sent to the client, control the transceiver unit 401 to receive a processing request submitted by the user on the visual interface; the processing request includes the metadata to be operated and operation instructions;
  • the transceiver unit 401 is controlled to receive the processing result returned by the calculation engine, and the processing result is added to the metadata model and then sent to the client.
  • processing unit 402 is specifically configured to:
  • the result set in the second format is a result set that is generated when each calculation engine processes historical tasks and is converted into a unified format
  • the result set in the second format is converted into the result set in the first format.
  • processing unit 402 is specifically configured to:
  • the result set of the third format is converted into the result set of the second format.
  • the present application also provides a computing device.
  • the computing device includes at least one processor 520 for implementing the method in FIG. 2 provided by the embodiment of the present application. Any method.
  • the computing device 500 may also include at least one memory 530 for storing program instructions and/or data.
  • the memory 530 and the processor 520 are coupled.
  • the coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units, or modules, and may be in electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.
  • the processor 520 may cooperate with the memory 530 to operate.
  • the processor 520 may execute program instructions stored in the memory 530. At least one of the at least one memory may be included in the processor.
  • the steps of the above method can be completed by hardware integrated logic circuits in the processor or instructions in the form of software.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware processor, or executed and completed by a combination of hardware and software modules in the processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. In order to avoid repetition, it will not be described in detail here.
  • the processor in the embodiment of the present application may be an integrated circuit chip with signal processing capability.
  • the steps of the foregoing method embodiments can be completed by hardware integrated logic circuits in the processor or instructions in the form of software.
  • the above-mentioned processor may be a general-purpose processor, a digital signal processing circuit (digital signal processor, DSP), a dedicated integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other Programming logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing circuit
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • Programming logic devices discrete gates or transistor logic devices, discrete hardware components.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
  • the memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), and electronic Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • the volatile memory may be random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • static random access memory static random access memory
  • dynamic RAM dynamic random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM
  • enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
  • serial link DRAM SLDRAM
  • direct rambus RAM direct rambus RAM
  • the computing device 500 may further include a communication interface 510 for communicating with other devices through a transmission medium, so that the apparatus used in the computing device 500 can communicate with other devices.
  • the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.
  • the transceiver when the communication interface is a transceiver, the transceiver may include an independent receiver and an independent transmitter; it may also be a transceiver with integrated transceiver functions, or an interface circuit.
  • the computing device 500 may also include a communication line 540.
  • the communication interface 510, the processor 520, and the memory 530 may be connected to each other through a communication line 540;
  • the communication line 540 may be a peripheral component interconnection standard (peripheral component interconnect, PCI for short) bus or an extended industry standard architecture (extended industry standard architecture) , Referred to as EISA) bus and so on.
  • the communication line 540 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used to indicate in FIG. 5, but it does not mean that there is only one bus or one type of bus.
  • embodiments of the present invention also provide a non-transitory computer-readable storage medium.
  • the non-transitory computer-readable storage medium stores computer instructions. When it runs on a computer, the computer executes the calculation based on the above.
  • the interface processing method of the engine is not limited to any one of the above.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a calculation program stored on a non-transitory computer-readable storage medium.
  • the computer program includes program instructions. When executed by a computer, the computer is caused to execute the aforementioned interface processing method based on the calculation engine.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种基于计算引擎的界面处理方法及装置，其中，方法包括：接收客户端发送的显示请求（201），根据显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集（202），生成记录有第一格式的结果集的第一任务，将第一任务提交至计算引擎（203），接收计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型（204），将元数据模型发送给客户端，以使客户端以可视化界面显示（205）。该技术方案用以提高结果集的可读性，方便操作人员的后续开发。

Description

一种基于计算引擎的界面处理方法及装置
相关申请的交叉引用
本申请要求在2019年07月17日提交中国专利局、申请号为201910645736.7、申请名称为“一种基于计算引擎的界面处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明实施例涉及金融科技(Fintech)领域，尤其涉及一种基于计算引擎的界面处理方法及装置。
背景技术
随着计算机技术的发展，越来越多的技术(例如：区块链、云计算或大数据)应用在金融领域，传统金融业正在逐步向金融科技转变，大数据技术也不例外，但由于金融、支付行业的安全性、实时性要求，也对大数据技术提出的更高的要求。
Apache Spark提供的两种基于命令行的处理交互方式虽然足够灵活，但在金融企业应用中面临诸如部署、安全等问题。为解决上述问题，现有技术中引入Livy，Livy是一个基于Spark的开源REST服务，可以通过代码片段向Spark提交任务，并同步或者异步地获取任务执行的结果集，以及对Spark contex进行管理，从而简化了Spark与一些应用程序的交互，并实现Spark被运用到一些web应用上。
但现有技术中，Livy将任务执行的结果集直接返回至操作界面，可读性差，且不利于后续工作人员的开发。
发明内容
本发明实施例提供一种基于计算引擎的界面处理方法及装置，用以提高结果集的可读性，方便操作人员的后续开发。
本发明实施例提供的一种基于计算引擎的界面处理方法，包括：
接收客户端发送的显示请求；
根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集；所述结果集是各计算引擎处理历史任务时生成的；
生成记录有所述第一格式的结果集的第一任务，将所述第一任务提交至所述计算引擎；
接收所述计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型；
将所述元数据模型发送给所述客户端，以使所述客户端以可视化界面显示。
上述技术方案中，各计算引擎处理历史任务时生成的结果集存储在数据库中，任务执行器可以在接收到客户端发送的显示请求后，根据显示请求中结果集标识确定对应格式的结果集，并将该结果集对应的第一任务提交至计算引擎中，计算引擎根据结果集生成临时视图并返回临时视图的元数据，任务执行器进而根据临时视图的元数据生成元数据模型，并在客户端中以可视化界面的方式显示，数据的可读性强，且工作人员可以根据显示的界面执行下一步操作，有利于工作人员根据上一步的结果集数据进行下一步操作。进一步的，将各计算引擎处理历史任务时生成的结果集存储在数据库中，有利于中间数据或结果集的 存储，在下一次提交任务时，无需提交包括上一步操作的任务，可以直接从数据库中调用结果集数据，提高计算引擎的运行效率。
可选的，在所述将所述元数据模型发送给所述客户端之后，还包括：
接收用户在所述可视化界面提交的处理请求；所述处理请求包括待操作的元数据及操作指令；
确定所述待操作的元数据对应的临时视图；
根据所述处理请求和所述待操作的元数据对应的临时视图，生成第二任务，所述第二任务与所述第一任务基于同一个上下文；
将所述第二任务提交至所述计算引擎；
接收所述计算引擎返回的处理结果，并将所述处理结果添加至所述元数据模型后发送给所述客户端。
上述技术方案中，任务执行器不仅可以将之前存储在数据库中的结果集的数据在客户端进行可视化显示，还可以针对该结果集生成第二任务，并将该第二任务提交至计算引擎进行下一步操作后，并将操作后生成的结果集显示在客户端界面，工作人员可以结合客户端的可视化界面进行开发操作，结果集数据的可读性强，且提高工作人员的开发效率。
可选的，所述根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集，包括：
根据所述显示请求中结果集标识，确定第二格式的结果集的存储路径；所述第二格式的结果集是将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集；
根据所述存储路径，从数据库中确定出所述第二格式的结果集；
根据所述显示请求中所述计算引擎的标识，将所述第二格式的结果集转换成所述第一格式的结果集。
可选的，所述将计算引擎处理历史任务时生成的结果集转换为统一格式的结果集，包括：
接收所述计算引擎处理历史任务时生成的第三格式的结果集；
根据所述第三格式的结果集，确定所述第三格式的结果集的格式化类；
根据所述第三格式的结果集的格式化类，将所述第三格式的结果集转换为所述第二格式的结果集。
上述技术方案中，将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集存储在数据库中，方便采用其它计算引擎对存储在数据库中的结果集进行操作时，读取该统一格式的结果集并将统一格式的结果集转换成对应计算引擎的格式。
相应的，本发明实施例还提供了一种基于计算引擎的界面处理装置，包括：
收发单元，用于接收客户端发送的显示请求；
处理单元，用于根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集；所述结果集是各计算引擎处理历史任务时生成的；
所述处理单元还用于，生成记录有所述第一格式的结果集的第一任务，将所述第一任务提交至所述计算引擎；
所述收发单元还用于，接收所述计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型；以及将所述元数据模型发送给所述客户端，以使所述客户端以可视化界面显示。
可选的，所述处理单元还用于：
在所述将所述元数据模型发送给所述客户端之后，控制所述收发单元接收用户在所述可视化界面提交的处理请求；所述处理请求包括待操作的元数据及操作指令；
确定所述待操作的元数据对应的临时视图；
根据所述处理请求和所述待操作的元数据对应的临时视图，生成第二任务，所述第二任务与所述第一任务基于同一个上下文；
将所述第二任务提交至所述计算引擎；
控制所述收发单元接收所述计算引擎返回的处理结果，并将所述处理结果添加至所述元数据模型后发送给所述客户端。
可选的，所述处理单元具体用于：
根据所述显示请求中结果集标识，确定第二格式的结果集的存储路径；所述第二格式的结果集是将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集；
根据所述存储路径，从数据库中确定出所述第二格式的结果集；
根据所述显示请求中所述计算引擎的标识，将所述第二格式的结果集转换成所述第一格式的结果集。
可选的，所述处理单元具体用于：
接收所述计算引擎处理历史任务时生成的第三格式的结果集；
根据所述第三格式的结果集，确定所述第三格式的结果集的格式化类；
根据所述第三格式的结果集的格式化类，将所述第三格式的结果集转换为所述第二格式的结果集。
相应的，本发明实施例还提供了一种计算设备，包括：
处理器、存储器、通信接口；其中，处理器、存储器与通信接口之间通过总线连接；
所述处理器，用于读取所述存储器中的程序，执行上述基于计算引擎的界面处理方法；
所述存储器，用于存储一个或多个可执行程序，可以存储所述处理器在执行操作时所使用的数据。
相应的，本发明实施例还提供了一种非暂态计算机可读存储介质，非暂态计算机可读存储介质中存储计算机指令，当其在计算机上运行时，使得计算机执行上述基于计算引擎的界面处理方法。
相应的，本发明实施例还提供一种包含指令的计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行上述基于计算引擎的界面处理方法。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种系统架构的示意图；
图2为本发明实施例提供的一种界面处理方法的流程示意图；
图3为本发明实施例提供的另一种界面处理方法的流程示意图；
图4为本发明实施例提供的一种界面处理装置的结构示意图；
图5为本申请提供的一种计算设备的结构示意图。
具体实施方式
为了使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明作进一步地详细描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。
图1示例性的示出了本发明实施例提供基于计算引擎的界面处理方法所适用的系统架构，该系统架构可以包括客户端100、执行控制器200、任务执行器300和计算引擎400。
其中，客户端100，作为用户编写开发语言的终端，提供编辑、执行、终止等基本交互界面，且向用户提供可视化的图表。
执行控制器200，作为服务端与客户端100直接交互，负责接收用户提交的代码与指令，维护用户的上下文状态与任务执行器的状态信息，并根据当前状态处理用户请求，使得用户提交的任务得到及时执行或向用户提供相关反馈信息。
任务执行器300用于将执行控制器200的任务提交至计算引擎400，并将计算引擎400的执行结果反馈。
计算引擎400可以是Spark、Hive和Hbase等。
基于上述描述，图2示例性的示出了本发明实施例提供的一种基于计算引擎的界面处理方法的流程，该流程可以由基于计算引擎的界面处理装置执行，该装置可以位于任务执行器中，可以是该任务执行器。
如图2所示，该流程具体包括：
步骤201，接收客户端发送的显示请求。
步骤202，根据显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集。
步骤203，生成记录有第一格式的结果集的第一任务，将第一任务提交至计算引擎。
步骤204，接收计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型。
步骤205，将元数据模型发送给客户端，以使客户端以可视化界面显示。
需要说明的是，结果集是各计算引擎处理历史任务时生成的，具体的，任务执行器将任务提交至计算引擎，计算引擎返回执行结果，该执行结果即计算引擎处理历史任务时生成的。进一步的，该执行结果可以理解为具有特定格式的结果集，该结果集的格式由计算引擎的类型确定，如，计算引擎是Spark，则Spark引擎返回的结果集格式是可以被Spark引擎处理的数据格式。
各计算引擎在处理历史任务时生成特定格式的结果集，将结果集返回至任务执行器，本发明实施例中，为方便将各计算引擎的结果集进行存储，任务执行器中记录有各计算引擎的结果集对应的格式化类，任务执行器在接收到计算引擎处理历史任务时生成的结果集后，会根据该结果集确定该结果集的格式化类，并根据该结果集的格式化类，将该结果集转换为统一格式。任务执行器与数据库连接，在将结果集转换为统一格式后，可以将该具有统一格式的结果集存储至数据库中。数据库可以是分布式数据库，可以为HDFS(Hadoop Distributed File System，分布式文件系统)。
各计算引擎在处理历史任务时生成的结果集相当于第三格式的结果集，数据库中存储 的统一格式的结果集相当于第二格式的结果集。上述实施例中，又可以理解为，任务执行器接收计算引擎处理历史任务时生成的第三格式的结果集；根据第三格式的结果集，确定第三格式的结果集的格式化类；根据第三格式的结果集的格式化类，将第三格式的结果集解析为第二格式的结果集。
一种具体的实现方式中，任务执行器中记录有所有开发语言对应的一个结果集格式化类。任务执行器根据用户提交的开发语言类型，找到该语言的结果集对应的格式化类，并根据该格式化类解析原始的结果集，并确定出该原始的结果集中的元数据信息和元数据的值，其中，元数据信息包括字段、字段类型、排序方式等。任务执行器将解析后的元数据信息转换成预设格式，如将结果集中python的numbe转换成double。将转换成预设格式的结果集存储至数据库中，以使得后续的数据读取。
在读取过程中，如步骤201和步骤202中，任务执行器接收客户端发送的显示请求，并根据显示请求中的结果集标识，确定用于计算引擎处理的第一格式的结果集。具体的，显示请求中记录有结果集标识，任务执行器可以根据显示请求中结果集标识，确定第二格式的结果集的存储路径，此处，第二格式即上述存储在数据库中的结果集的统一格式。任务执行器根据第二格式的结果集的存储路径，从数据库中确定出第二格式的结果集，并根据显示请求中计算引擎的标识，将第二格式的结果集转换成第一格式的结果集。也就是说，任务执行器在读取到统一格式的结果集后，会将该结果集在转化成用于计算引擎处理的格式的结果集。
示例性的，任务执行器接收客户端发送的显示请求，该显示请求用于指示根据结果集生成任务后提交给Spark执行，则任务执行器在确定出第二格式的结果集后，会将该第二格式的结果集转换成Spark格式的结果集后，生成Spark任务后提交。实际应用中，任务执行器可以根据第二格式的结果集的存储路径，从HDFS中读取第二格式的结果集，并将其在内存中以ResultSetReader对象进行展示。该对象持有第二格式的结果集的所有元数据信息和元数据的值。然后，任务执行器将第二格式的结果集的元数据信息(字段名称和字段类型)转换成为Spark支持的元数据类型(StructType)。其中，字段类型通过查询一个一对一的对应关系来实现转换，每个第二格式的字段类型均有对应的Spark DataType。字段名称直接沿用第二格式中的字段名称不变。任务执行器将第二格式的元数据的值转换成Spark的值类型(Row)，第二格式的结果集对象中，每一行数值都是一个数组，而Spark Row的构造函数接受值的数组作为参数，可以直接遍历结果集对象的所有行，将数组传递给Row的构造方法，获得相关的一系列Row对象。得到的StructType和Row对象，即为Spark格式的结果集。
上述技术方案中，任务执行器将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集存储在数据库中，方便采用其它计算引擎对存储在数据库中的结果集进行操作时，读取该统一格式的结果集并将统一格式的结果集转换成对应计算引擎的格式。例如，任务执行器将Spark格式的结果集按照统一格式存储在数据库中，随后可以读取该统一格式的结果集并转换成Hive格式的结果集，用于在Hive中计算。通过该方式可以实现将任务提交给Spark、Hive、Shell等执行环境。
在确定出用于计算引擎处理的格式的结果集(第一格式的结果集)后，任务执行器可以生成记录有第一格式的结果集的第一任务，并将该第一任务提交至计算引擎，计算引擎根据该第一任务在计算引擎中创建临时视图，并将临时视图中的元数据反馈至任务执行器， 任务执行器根据元数据构建可视化的元数据模型，并将构建好的可视化的元数据模型发送至客户端，客户端进行可视化界面的显示。此处，临时视图中的元数据可以理解为上述元数据类型，包括字段类型、字段名称等。
本发明实施例中，计算引擎可以是Spark引擎，在任务执行器确定出Spark格式的结果集后，可以先判断当前Spark引擎中是否存在与第一任务对应的Spark context，若没有，则启动一个Spark context，若有，则直接将Spark格式的结果集传入Spark的createDataFrame方法，得到一个Dataframe后，调用Spark提供的createOrReplaceTempView方法，即可将创建临时视图的代码片段提交给Spark执行，从而完成在Spark context中创建临时视图。
进一步的，Spark引擎将临时视图中的元数据反馈至任务执行器，任务执行器创建元数据模型，并可以采用json格式进行存储。可选的，元数据模型中包含了所有的字段和字段类型信息(column和datatype的数组)，并能够用来承载后面定义的指标、维度和筛选条件等信息(也通过对象的数组来实现)。元数据模型会以json的方式返回给客户端进行解析，客户端可以将元数据模型拥有的字段都展现出来。
此外，在任务执行器将元数据模型发送给客户端之后，客户端显示元数据模型，用户可以在客户端的显示界面上执行运算操作，用户可以在可视化界面提交处理请求，客户端将用户提交的处理请求发送至任务执行器，该处理请求中可以包括待操作的元数据及操作指令，任务执行器根据处理请求从第一任务对应的上下文中确定待操作的元数据对应的临时视图，并根据处理请求和待操作的元数据对应的临时视图，生成第二任务，将第二任务提交至计算引擎；计算引擎根据第二任务执行运算操作，具体的，计算引擎从临时视图中确定元数据对应的值，并将数据对应的值执行上述操作指令得到处理结果，也就是得到第二任务对应的结果集。任务执行器接收计算引擎返回的处理结果，并将处理结果添加至元数据模型后发送给客户端。
一种具体的实现方式中，客户端界面可以提供可视化按钮，当任务执行器将计算引擎的结果集返回后，用户可以通过客户端界面确定结果集是否符合用户需求，并在确定符合需求后，通过可视化按钮将结果集提交给任务执行器，进一步的，任务执行器执行上述操作，将结果集对应的元数据模型显示在客户端界面上，客户端界面上可以根据元数据模型显示各种字段，并组成指标、维度和筛选条件等信息，用户可以在客户端上选择针对元数据的运算方式，比如要对某个字段做聚合运算，便将其拖入指标区域内，选择聚合运算的方式后保存。任务执行器可以根据元数据和运算方式，拼接成SQL(Structured Query Language，结构化查询语言)代码片段，比如对A字段做求和，就转换成为sum(A)代码片段。任务执行器将SQL代码片段提交至计算引擎，计算引擎根据SQL代码片段生成处理结果，任务执行器将处理结果填入元数据模型发送至客户端，客户端根据用户选择的图表模式，将数据填充进图表后进行渲染，进行最终的可视化展示。
一种具体的实现方式中，可以设置可视化服务模块，该可视化服务模块可以设置在上述任务执行器中，以完成任务执行器的部分任务。可视化服务模块还可以设置在执行控制器中或者是一个独立的服务器，在此不做限制。基于可视化服务模块，可以如图3示出的另一种可视化流程图，该流程中还涉及客户端、任务执行器和计算引擎。
步骤301，客户端生成可视化请求；
步骤302，客户端将可视化请求发送至可视化服务模块；
步骤303，可视化服务模块根据可视化请求创建临时视图任务；
步骤304，可视化服务模块将临时视图任务经任务执行器提交至计算引擎；
步骤305，计算引擎创建临时视图；
步骤306，可视化服务模块将查询临时视图请求发送至任务执行器；
步骤307，任务执行器从计算引擎中查询临时视图的元数据；
步骤308，任务执行器根据临时视图的元数据生成元数据模型；
步骤309，任务执行器通过可视化模块将元数据模型发送至客户端；
步骤310，客户端显示可拖拽的元数据前端组件；
步骤311，客户端界面将指标、维度等配置信息发送至可视化服务模块；
步骤312，可视化服务模块拼接SQL语句，生成查询任务；
步骤313，可视化服务模块将查询任务经任务执行器发送至计算引擎；
步骤314，计算引擎根据查询任务生成查询结果集；
步骤315，计算引擎将查询结果集发送至任务执行器；
步骤316，任务执行器将查询结果格式化后生成统一格式的结果集；
步骤317，任务执行器将统一格式的结果集填入元数据模型，作为待显示数据；
步骤319，任务执行器将待显示数据经可视化服务模块发送至客户端界面；
步骤320，客户端界面渲染成图表。
上述技术方案中，任务执行器将各计算引擎处理历史任务时生成的结果集存储在数据库中，任务执行器可以在接收到客户端发送的显示请求后，根据显示请求中结果集标识确定对应格式的结果集，并将该结果集对应的第一任务提交至计算引擎中，计算引擎根据结果集生成临时视图并返回临时视图的元数据，任务执行器进而根据临时视图的元数据生成元数据模型，并在客户端中以可视化界面的方式显示，数据的可读性强，且工作人员可以根据显示的界面执行下一步操作，有利于工作人员根据上一步的结果集数据进行下一步操作。进一步的，将各计算引擎处理历史任务时生成的结果集存储在数据库中，有利于中间数据或结果集的存储，在下一次提交任务时，无需提交包括上一步操作的任务，可以直接从数据库中调用结果集数据，提高计算引擎的运行效率。
基于统一发明构思，图4示例性的示出了本发明实施例提供的一种基于计算引擎的界面处理装置的结构，该装置可以执行基于计算引擎的界面处理方法的流程。
该装置包括：
收发单元401，用于接收客户端发送的显示请求；
处理单元402，用于根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集；所述结果集是各计算引擎处理历史任务时生成的；
所述处理单元402还用于，生成记录有所述第一格式的结果集的第一任务，将所述第一任务提交至所述计算引擎；
所述收发单元401还用于，接收所述计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型；以及将所述元数据模型发送给所述客户端，以使所述客户端以可视化界面显示。
可选的，所述处理单元402还用于：
在所述将所述元数据模型发送给所述客户端之后，控制所述收发单元401接收用户在所述可视化界面提交的处理请求；所述处理请求包括待操作的元数据及操作指令；
确定所述待操作的元数据对应的临时视图；
根据所述处理请求和所述待操作的元数据对应的临时视图，生成第二任务，所述第二任务与所述第一任务基于同一个上下文；
将所述第二任务提交至所述计算引擎；
控制所述收发单元401接收所述计算引擎返回的处理结果，并将所述处理结果添加至所述元数据模型后发送给所述客户端。
可选的，所述处理单元402具体用于：
根据所述显示请求中结果集标识，确定第二格式的结果集的存储路径；所述第二格式的结果集是将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集；
根据所述存储路径，从数据库中确定出所述第二格式的结果集；
根据所述显示请求中所述计算引擎的标识，将所述第二格式的结果集转换成所述第一格式的结果集。
可选的，所述处理单元402具体用于：
接收所述计算引擎处理历史任务时生成的第三格式的结果集；
根据所述第三格式的结果集，确定所述第三格式的结果集的格式化类；
根据所述第三格式的结果集的格式化类，将所述第三格式的结果集转换为所述第二格式的结果集。
基于与上述图2所示的方法相同的构思，本申请还提供一种计算设备，如图5所示，该计算设备包括至少一个处理器520，用于实现本申请实施例提供的图2中任一方法。
计算设备500还可以包括至少一个存储器530，用于存储程序指令和/或数据。存储器530和处理器520耦合。本申请实施例中的耦合是装置、单元或模块之间的间接耦合或通信连接，可以是电性，机械或其它的形式，用于装置、单元或模块之间的信息交互。处理器520可能和存储器530协同操作。处理器520可能执行存储器530中存储的程序指令。所述至少一个存储器中的至少一个可以包括于处理器中。
在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
应注意，本申请实施例中的处理器可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理电路(digital signal processor，DSP)、专用集成芯片(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的 步骤。
可以理解，本申请实施例中的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
计算设备500还可以包括通信接口510，用于通过传输介质和其它设备进行通信，从而用于计算设备500中的装置可以和其它设备进行通信。在本申请实施例中，通信接口可以是收发器、电路、总线、模块或其它类型的通信接口。在本申请实施例中，通信接口为收发器时，收发器可以包括独立的接收器、独立的发射器；也可以集成收发功能的收发器、或者是接口电路。
计算设备500还可以包括通信线路540。其中，通信接口510、处理器520以及存储器530可以通过通信线路540相互连接；通信线路540可以是外设部件互连标准(peripheral component interconnect，简称PCI)总线或扩展工业标准结构(extended industry standard architecture，简称EISA)总线等。所述通信线路540可以分为地址总线、数据总线、控制总线等。为便于表示，图5中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
基于同一发明构思，本发明实施例还提供了一种非暂态计算机可读存储介质，非暂态计算机可读存储介质中存储计算机指令，当其在计算机上运行时，使得计算机执行上述基于计算引擎的界面处理方法。
基于同一发明构思，本申请实施例提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行上述基于计算引擎的界面处理方法。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个 方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。

Claims (11)

  1. 一种基于计算引擎的界面处理方法，其特征在于，包括：
    接收客户端发送的显示请求；
    根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集；所述结果集是各计算引擎处理历史任务时生成的；
    生成记录有所述第一格式的结果集的第一任务，将所述第一任务提交至所述计算引擎；
    接收所述计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型；
    将所述元数据模型发送给所述客户端，以使所述客户端以可视化界面显示。
  2. 如权利要求1所述的方法，其特征在于，在所述将所述元数据模型发送给所述客户端之后，还包括：
    接收用户在所述可视化界面提交的处理请求；所述处理请求包括待操作的元数据及操作指令；
    确定所述待操作的元数据对应的临时视图；
    根据所述处理请求和所述待操作的元数据对应的临时视图，生成第二任务，所述第二任务与所述第一任务基于同一个上下文；
    将所述第二任务提交至所述计算引擎；
    接收所述计算引擎返回的处理结果，并将所述处理结果添加至所述元数据模型后发送给所述客户端。
  3. 如权利要求1所述的方法，其特征在于，所述根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集，包括：
    根据所述显示请求中结果集标识，确定第二格式的结果集的存储路径；所述第二格式的结果集是将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集；
    根据所述存储路径，从数据库中确定出所述第二格式的结果集；
    根据所述显示请求中所述计算引擎的标识，将所述第二格式的结果集转换成所述第一格式的结果集。
  4. 如权利要求3所述的方法，其特征在于，所述将计算引擎处理历史任务时生成的结果集转换为统一格式的结果集，包括：
    接收所述计算引擎处理历史任务时生成的第三格式的结果集；
    根据所述第三格式的结果集，确定所述第三格式的结果集的格式化类；
    根据所述第三格式的结果集的格式化类，将所述第三格式的结果集转换为所述第二格式的结果集。
  5. 一种基于计算引擎的界面处理装置，其特征在于，包括：
    收发单元，用于接收客户端发送的显示请求；
    处理单元，用于根据所述显示请求中结果集标识，确定用于计算引擎处理的第一格式的结果集；所述结果集是各计算引擎处理历史任务时生成的；
    所述处理单元还用于，生成记录有所述第一格式的结果集的第一任务，将所述第一任务提交至所述计算引擎；
    所述收发单元还用于，接收所述计算引擎反馈的临时视图对应的元数据并构建可视化的元数据模型；以及将所述元数据模型发送给所述客户端，以使所述客户端以可视化界面 显示。
  6. 如权利要求5所述的装置，其特征在于，所述处理单元还用于：
    在所述将所述元数据模型发送给所述客户端之后，控制所述收发单元接收用户在所述可视化界面提交的处理请求；所述处理请求包括待操作的元数据及操作指令；
    确定所述待操作的元数据对应的临时视图；
    根据所述处理请求和所述待操作的元数据对应的临时视图，生成第二任务，所述第二任务与所述第一任务基于同一个上下文；
    将所述第二任务提交至所述计算引擎；
    控制所述收发单元接收所述计算引擎返回的处理结果，并将所述处理结果添加至所述元数据模型后发送给所述客户端。
  7. 如权利要求5所述的装置，其特征在于，所述处理单元具体用于：
    根据所述显示请求中结果集标识，确定第二格式的结果集的存储路径；所述第二格式的结果集是将各计算引擎处理历史任务时生成的结果集转换为统一格式的结果集；
    根据所述存储路径，从数据库中确定出所述第二格式的结果集；
    根据所述显示请求中所述计算引擎的标识，将所述第二格式的结果集转换成所述第一格式的结果集。
  8. 如权利要求7所述的装置，其特征在于，所述处理单元具体用于：
    接收所述计算引擎处理历史任务时生成的第三格式的结果集；
    根据所述第三格式的结果集，确定所述第三格式的结果集的格式化类；
    根据所述第三格式的结果集的格式化类，将所述第三格式的结果集转换为所述第二格式的结果集。
  9. 一种计算设备，其特征在于，包括处理器、存储器、通信接口，其中处理器、存储器与通信接口之间通过总线连接；
    所述处理器，用于读取所述存储器中的程序，执行权利要求1至4任一所述方法；
    所述存储器，用于存储一个或多个可执行程序，以及存储所述处理器在执行操作时所使用的数据。
  10. 一种非暂态计算机可读存储介质，其特征在于，所述非暂态计算机可读存储介质存储计算机指令，所述计算机指令用于使所述计算机执行权利要求1至4任一所述方法。
  11. 一种计算机程序产品，其特征在于，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行权利要求1至4任一所述方法。
PCT/CN2020/102226 2019-07-17 2020-07-15 一种基于计算引擎的界面处理方法及装置 Ceased WO2021008570A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910645736.7A CN110377621B (zh) 2019-07-17 2019-07-17 一种基于计算引擎的界面处理方法及装置
CN201910645736.7 2019-07-17

Publications (1)

Publication Number Publication Date
WO2021008570A1 true WO2021008570A1 (zh) 2021-01-21

Family

ID=68253653

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/102226 Ceased WO2021008570A1 (zh) 2019-07-17 2020-07-15 一种基于计算引擎的界面处理方法及装置

Country Status (2)

Country Link
CN (1) CN110377621B (zh)
WO (1) WO2021008570A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742040A (zh) * 2021-08-09 2021-12-03 广州市易工品科技有限公司 基于可视化界面快速生成分布式批处理任务的方法和装置
CN114490567A (zh) * 2021-12-23 2022-05-13 深圳太极数智技术有限公司 数据交换方法、系统、设备及存储介质
CN114579660A (zh) * 2022-03-12 2022-06-03 广州车行易科技股份有限公司 数据可视化展示方法以及相关设备
CN116450698A (zh) * 2023-01-17 2023-07-18 中国联合网络通信集团有限公司 记录型数据处理方法、装置、设备及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110377621B (zh) * 2019-07-17 2023-12-01 深圳前海微众银行股份有限公司 一种基于计算引擎的界面处理方法及装置
CN111831274B (zh) * 2020-06-29 2023-10-20 中孚信息股份有限公司 一种元数据解析引擎配置系统、方法、终端及可读存储介质
CN112256710B (zh) * 2020-09-30 2022-12-06 中孚安全技术有限公司 一种基于元数据的数据统计分析图表生成系统、方法及设备
CN113722337B (zh) * 2021-11-03 2022-06-10 深圳市信润富联数字科技有限公司 业务数据确定方法、装置、设备及存储介质
CN116932575B (zh) * 2023-09-12 2023-12-15 长城证券股份有限公司 基于Spark的跨数据源操作方法、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102567026A (zh) * 2011-12-28 2012-07-11 用友软件股份有限公司 移动应用构建系统和移动应用构建方法
US20120224745A1 (en) * 2011-03-03 2012-09-06 International Business Machines Corporation Evaluation of graphical output of graphical software applications executing in a computing environment
CN109345377A (zh) * 2018-09-28 2019-02-15 北京九章云极科技有限公司 一种数据实时处理系统及数据实时处理方法
CN109445792A (zh) * 2018-11-05 2019-03-08 用友网络科技股份有限公司 界面构建方法、装置及计算机可读存储介质
CN110377621A (zh) * 2019-07-17 2019-10-25 深圳前海微众银行股份有限公司 一种基于计算引擎的界面处理方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120224745A1 (en) * 2011-03-03 2012-09-06 International Business Machines Corporation Evaluation of graphical output of graphical software applications executing in a computing environment
CN102567026A (zh) * 2011-12-28 2012-07-11 用友软件股份有限公司 移动应用构建系统和移动应用构建方法
CN109345377A (zh) * 2018-09-28 2019-02-15 北京九章云极科技有限公司 一种数据实时处理系统及数据实时处理方法
CN109445792A (zh) * 2018-11-05 2019-03-08 用友网络科技股份有限公司 界面构建方法、装置及计算机可读存储介质
CN110377621A (zh) * 2019-07-17 2019-10-25 深圳前海微众银行股份有限公司 一种基于计算引擎的界面处理方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742040A (zh) * 2021-08-09 2021-12-03 广州市易工品科技有限公司 基于可视化界面快速生成分布式批处理任务的方法和装置
CN113742040B (zh) * 2021-08-09 2024-04-19 广州市易工品科技有限公司 基于可视化界面快速生成分布式批处理任务的方法和装置
CN114490567A (zh) * 2021-12-23 2022-05-13 深圳太极数智技术有限公司 数据交换方法、系统、设备及存储介质
CN114579660A (zh) * 2022-03-12 2022-06-03 广州车行易科技股份有限公司 数据可视化展示方法以及相关设备
CN116450698A (zh) * 2023-01-17 2023-07-18 中国联合网络通信集团有限公司 记录型数据处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN110377621B (zh) 2023-12-01
CN110377621A (zh) 2019-10-25

Similar Documents

Publication Publication Date Title
WO2021008570A1 (zh) 一种基于计算引擎的界面处理方法及装置
CN108519967B (zh) 图表可视化方法、装置、终端和存储介质
CN112148509A (zh) 数据处理方法、装置、服务器及计算机可读存储介质
CN110705036B (zh) 一种核电设计任务管理系统，方法和存储介质
JP2022028881A (ja) 広告を自動的に生成する方法、装置、機器及びコンピュータ可読記憶媒体
WO2016082468A1 (zh) 数据图形化方法、装置及数据库服务器
CN110688541A (zh) 一种报表数据的查询方法、装置、存储介质及电子设备
CN106341444A (zh) 数据访问方法和装置
US12602395B2 (en) Apparatus and method for filtering visualizations from or across different analytics platforms
CN119557330A (zh) 基于大语言模型的数据查询方法、系统
CN115857929B (zh) 资源数据处理方法、装置、计算机设备和存储介质
KR102172138B1 (ko) 분산 컴퓨팅 프레임워크 및 분산 컴퓨팅 방법
US20150169515A1 (en) Data driven synthesizer
CN110347386A (zh) 一种基于sql代码编辑的数据可视化分析的方法、装置和电子设备
CN112163948B (zh) 一种分润计算方法、系统、设备及存储介质
CN111966349B (zh) 基于bs架构程序开发的方法、装置和计算机设备
CN118034672B (zh) 一种基于Spark的可视化数据处理分析方法及系统
CN118917296A (zh) 基于大语言模型的报告生成方法、设备及存储介质
CN115618830B (zh) 报表生成方法、系统、装置、电子设备和存储介质
CN116662461A (zh) 一种多数据源的数据可视化的方法及系统
US10360701B2 (en) Integrated visualization
CN115600038A (zh) 页面渲染方法、装置、设备及介质
US20170052998A1 (en) System and method for efficient database transactions
CN116225381A (zh) 一种用于数据可视化的多数据源接入方法及系统
CN114896487A (zh) 前端与区块链交互的区块链浏览器实现方法、系统及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20840534

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20840534

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20840534

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.09.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20840534

Country of ref document: EP

Kind code of ref document: A1