CN109726216A - A kind of data processing method and processing system based on directed acyclic graph - Google Patents

A kind of data processing method and processing system based on directed acyclic graph Download PDF

Info

Publication number
CN109726216A
CN109726216A CN201811644309.9A CN201811644309A CN109726216A CN 109726216 A CN109726216 A CN 109726216A CN 201811644309 A CN201811644309 A CN 201811644309A CN 109726216 A CN109726216 A CN 109726216A
Authority
CN
China
Prior art keywords
node
data
directed acyclic
acyclic graph
point
Prior art date
Application number
CN201811644309.9A
Other languages
Chinese (zh)
Inventor
邱剑生
郭璐
胡宗星
Original Assignee
北京九章云极科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京九章云极科技有限公司 filed Critical 北京九章云极科技有限公司
Priority to CN201811644309.9A priority Critical patent/CN109726216A/en
Publication of CN109726216A publication Critical patent/CN109726216A/en

Links

Abstract

The present invention provides a kind of data processing method and processing system based on directed acyclic graph, this method comprises: the first user interface of display, the first user interface includes node, and node includes source point, operator and place point;In response to the selection operation to node, the icon of the destination node of selection is shown in the first display area of the first user interface;In response to the attended operation to destination node, it will be connected including source point, operator with the destination node of place point using node connecting line, and be created as directed acyclic graph;In directed acyclic graph, the data that source point obtains are input to operator by node connecting line, and by operator, treated that data are input to place point to export target data.The present invention can provide visual user interface, for creating the directed acyclic graph that can be used in static data processing or real time data processing, the computer programming sentence using profession is not needed, it is easy to operate.

Description

A kind of data processing method and processing system based on directed acyclic graph

Technical field

The present invention relates to big data processing field more particularly to a kind of data processing methods and place based on directed acyclic graph Reason system.

Background technique

Stream calculation is an important component in current data processing field.With traditional data processing system number Calculation processing difference is carried out again according to being stored in after hard disk or other storage services, stream calculation processing data incoming in real time and real When the value for embodying data, generally believe that the value of data flow lowers with the passing of time.

Existing stream calculation system uses SQL (Structured Query Language, structured query language) language Sentence generates corresponding DAG (Directed Acyclic Graph, directed acyclic graph), and being generated with directed acyclic graph includes data Data flow logic in directed acyclic graph, is distributed to different nodes, professional for user is wanted by the node for flowing logic Ask higher, and inconvenient.

Summary of the invention

In view of this, the present invention provides a kind of data processing method and processing system based on directed acyclic graph, for solving Certainly existing stream calculation system needs that the corresponding directed acyclic of data flow can be generated using the computer programming sentence of profession Figure, professional requirement is high, inconvenient problem.

In order to solve the above technical problems, the present invention provides a kind of data processing method based on directed acyclic graph, comprising:

Show that the first user interface, first user interface include the node for creating directed acyclic graph, the section Point includes the source point for obtaining the input data of data source, the operator for being handled input data and for exporting mesh Mark the place point of data;

In response to the selection operation to the node, the icon of the destination node of selection is shown in the first user interface In first display area;

It will include the source point, operator and place using node connecting line in response to the attended operation to the destination node The destination node connection of point, is created as directed acyclic graph;

Wherein, in the directed acyclic graph, the data that the source point obtains are input to described by the node connecting line Operator, and treated that data are input to the place point to export target data the operator.

Optionally, first user interface further includes the second display area, and second display area is for showing institute State node.

Optionally, the selection operation is to click the clicking operation of the node, alternatively, the node is drawn to described The drag operation of first display area.

Optionally, the destination node includes output anchor point and/or input anchor point, and the attended operation is from a mesh Mark the drag operation that the output anchor point of node is pulled to the input anchor point of another destination node.

Optionally, the method also includes:

In response to the configuration operation to the destination node or the node connecting line in first display area, show Show information configuration interface;

Based on the configuration information that user inputs at the information configuration interface, to the institute shown in first display area It states destination node or the node connecting line is configured.

Optionally, the configuration information of the destination node includes at least one of the following: title, description information, output field Structure and parameter configuration.

Optionally, the configuration information of the node connecting line includes at least one of the following: data filtering information and data point Stream information.

Optionally, the destination node includes output anchor point and/or input anchor point, in response to the company to the destination node Operation is connect, will include the steps that the source point, operator are connected with the destination node of place point using node connecting line, comprising:

It is shown based on default display mode and exports the matched input anchor point of anchor point with target;

Target input anchor point is connected to from target output anchor point using node connecting line.

Optionally, the method also includes:

In response to the preservation operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code saves.

Optionally, the method also includes:

In response to the publication operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code;

It is the corresponding data flow of Flink by the code conversion of the json format;

The data flow is run, stream operation is generated.

Optionally, the method also includes:

In response to the export operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code, and be compressed file export by the code conversion of the json format.

Optionally, the method also includes:

In response to the import operation of the directed acyclic graph to creation, the directed acyclic graph after file is converted will be imported and be shown in Current interface.

Optionally, the import operation in response to the directed acyclic graph to creation, it is oriented after file is converted by importing Acyclic figure is shown in current interface

In response to importing the import operation of the code of the json format of compression, the generation of the json format of the compression is decompressed Code;

It is directed acyclic graph by the code conversion of the json format after decompression, and covers the directed acyclic graph currently created.

Optionally, the method also includes:

In response to uploading the upload operation of operator, the operator of upload is added to the node for being used to create directed acyclic graph In.

Optionally, the method also includes:

In response to the debugging operations debugged to the stream operation, the stream operation is debugged, and show debugging As a result.

Optionally, the method also includes:

In response to checking operation to the log of the stream operation, the log of the stream operation is shown.

Optionally, the method also includes:

Operation is checked in response to the operating status to the stream operation, is obtained in the stream operation at least partly described The monitoring information of node, and show the monitoring information.

The present invention also provides a kind of data processing systems based on directed acyclic graph, comprising:

Display module, for showing the first user interface, first user interface includes for creating directed acyclic graph Node, the node includes the source point for obtaining the input data of data source, the calculation for being handled input data Son and the place point for exporting target data;

First execution module, in response to the selection operation to the node, the icon of the destination node of selection to be shown It is shown in the first display area of the first user interface;

Second execution module, for that will include using node connecting line in response to the attended operation to the destination node The source point, operator are connected with the destination node of place point, are created as directed acyclic graph;Wherein, in the directed acyclic graph, institute It states node connecting line and the data that the source point obtains is input to the operator, and treated that data are input to the operator The place point is to export target data.

Optionally, first user interface further includes the second display area, and second display area is for showing institute State node.

Optionally, the selection operation is to click the clicking operation of the node, alternatively, the node is drawn to described The drag operation of first display area.

Optionally, the destination node includes output anchor point and/or input anchor point, and the attended operation is from a mesh Mark the drag operation that the output anchor point of node is pulled to the input anchor point of another destination node.

Optionally, the system also includes:

Third execution module, in response to in first display area the destination node or the node connect The configuration of wiring operates, and shows information configuration interface;

4th execution module, the configuration information for being inputted based on user at the information configuration interface, to described first The destination node or the node connecting line shown in display area is configured.

Optionally, the configuration information of the destination node includes at least one of the following: title, description information, output field Structure and parameter configuration.

Optionally, the configuration information of the node connecting line includes at least one of the following: data filtering information and data point Stream information.

Optionally, the destination node includes that output anchor point and/or input anchor point, second execution module are also used to It is shown based on default display mode and exports the matched input anchor point of anchor point with target;It is exported using node connecting line from the target Anchor point is connected to target input anchor point.

Optionally, the system also includes:

5th execution module, for the preservation operation in response to the directed acyclic graph to creation, by the directed acyclic of creation The code that figure is converted into json format saves.

Optionally, the system also includes:

6th execution module, for the publication operation in response to the directed acyclic graph to creation, by the directed acyclic of creation Figure is converted into the code of json format;It is the corresponding data flow of Flink by the code conversion of the json format;Run the number According to stream, stream operation is generated.

Optionally, the system also includes:

7th execution module, for the export operation in response to the directed acyclic graph to creation, by the directed acyclic of creation Figure is converted into the code of json format, and is compressed file export by the code conversion of the json format.

Optionally, the system also includes:

8th execution module, for the import operation in response to the directed acyclic graph to creation, after file conversion will be imported Directed acyclic graph be shown in current interface.

Optionally, the system also includes:

8th execution module is further used for the import operation of the code of the json format in response to importing compression, Decompress the code of the json format of the compression;It is directed acyclic graph by the code conversion of the json format after decompression, and covers The directed acyclic graph currently created.

Optionally, the system also includes:

The operator of upload is added to for the upload operation in response to uploading operator and is used to create by the 9th execution module In the node of directed acyclic graph.

Optionally, the system also includes:

Tenth execution module, in response to the debugging operations debugged of stream operation, to the stream operation into Row debugging, and show debugging result.

Optionally, the system also includes:

11st execution module, for showing the stream operation in response to checking operation to the log of the stream operation Log.

Optionally, the system also includes:

12nd execution module obtains the stream for checking operation in response to the operating status to the stream operation To the monitoring information of at least partly described node in operation, and show the monitoring information.

The present invention also provides a kind of processing equipment, including processor, memory and it is stored on the memory and can be The computer program run on the processor, the computer program are realized above-mentioned based on oriented when being executed by the processor The step of data processing method of acyclic figure.

The present invention also provides a kind of computer readable storage medium, computer is stored on the computer readable storage medium The step of program, the computer program realizes the above-mentioned data processing method based on directed acyclic graph when being executed by processor.

The advantageous effects of the above technical solutions of the present invention are as follows:

In an embodiment of the present invention, it is capable of providing visual user interface, can be used in static data for creating The directed acyclic graph of processing or real time data processing does not need the computer programming sentence using profession, for the profession of user Property require it is lower and easy to operate.

Detailed description of the invention

Fig. 1 is the flow diagram of the data processing method based on directed acyclic graph of the embodiment of the present invention;

Fig. 2 is the schematic diagram of the first user interface provided in an embodiment of the present invention;

Fig. 3 is the first embodiment for the user interface that the present invention configures an operator;

Fig. 4 is the second embodiment for the user interface that the present invention configures an operator;

Fig. 5 is the schematic diagram at the interface of editor's application message of the embodiment of the present invention;

Fig. 6 is the schematic diagram that the Jar of the embodiment of the present invention assures the interface of reason;

Fig. 7 and Fig. 8 is the debugging window schematic diagram of the stream operation of the embodiment of the present invention;

Fig. 9 is the monitoring window schematic diagram of the stream operation of the embodiment of the present invention;

Figure 10 is the structural schematic diagram of the data processing system based on directed acyclic graph of the embodiment of the present invention;

Figure 11 is the structural schematic diagram of the processing equipment of the embodiment of the present invention.

Specific embodiment

In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention Attached drawing, the technical solution of the embodiment of the present invention is clearly and completely described.Obviously, described embodiment is this hair Bright a part of the embodiment, instead of all the embodiments.Based on described the embodiment of the present invention, ordinary skill Personnel's every other embodiment obtained, shall fall within the protection scope of the present invention.

To solve existing stream calculation system need that data flow pair can be generated using the computer programming sentence of profession The directed acyclic graph answered, professional requirement is high, inconvenient problem, referring to FIG. 1, Fig. 1 be the embodiment of the present invention based on The data processing method of the flow diagram of the data processing method of directed acyclic graph, the directed acyclic graph includes:

Step 11: the first user interface of display, first user interface includes the node for creating directed acyclic graph, The node includes the source point (source) for obtaining the input data of data source, the calculation for being handled input data Sub (operator) and the place point (sink) for exporting target data;

So-called directed acyclic graph is as group, arrangement, block chain and a kind of data structure.

Step 12: in response to the selection operation to the node, the icon of the destination node of selection being shown in the first use In first display area at family interface;

Step 13: will include the source point, calculation using node connecting line in response to the attended operation to the destination node Son is connected with the destination node of place point, is created as directed acyclic graph;Wherein, in the directed acyclic graph, the node connection The data that the source point obtains are input to the operator by line, and the operator treated data the are input to place point with Export target data.In an embodiment of the present invention, it is capable of providing visual user interface, can be used in static state for creating The directed acyclic graph of data processing or real time data processing does not need the computer programming sentence using profession, for user's It is professional require it is lower and easy to operate.

The citing of source point, operator and place point is illustrated separately below.

The source point indicates the source that the data of directed acyclic graph flow into, such as may include at least one of following: local File source (from data are read in local file), the source socket (data are read from socket), the source http (connect from http endpoint Receive data), the source JDBC is (with JDBC from database (mysql, DB (Database, database) 2) in read data), analogue data Source (interval output specified data according to the specified time), IBM MQ (reading data from WebSphere MQ) and Kafka Source (consumption data from specified Kafka theme).

The intermediate node of the operator representation directed acyclic graph, handles input data, such as may include following It is at least one: the classes such as converging operation, Http operation, machine learning, serializing operation, statistics, stream operation, map function and window Not.

Wherein, converging operation includes pure window polymerization, key polymerization and window polymerization;Pure window polymerization is by window polymerization note Record, when polymerization, can add major key, achieve the purpose that be polymerize based on certain keys;Key polymerization carries out field based on Key Converging operation, addition major key field name configuration need to configure polymerization Id, operator can according to the configuration in major key field name, into Row polymerization;Window polymerization can add major key by window record polymerization, when polymerization, reach based on certain keys and polymerize Purpose.Pure window polymerization can be understood as normal window polymerization, and window polymerization has been done on this basis patrols the fractionation of window Volume.Because the data in a window may be excessive, calculator memory is caused to be not enough to store the data in next window, that In window polymerization, understands and this window is split into different wickets, each independent calculated result of wicket, and it is final It merges.

Http operation includes that Http is inquired, and Http inquiry is by sending Http request data, the data that will acquire It stores in output field.Output field refers to that the column name of data, data are to pass in entire stream in the form of key:value Defeated, so each data all has one or more key, key is exactly to arrange name here.

Machine learning includes Drools rule, PMML model and Kalman filter;Drools rule uses customized Drools rule handles data, and the file of upload must be ended up with .jar, and the inside will include Drools rule text Part after file uploads successfully, can obtain file name as module name automatically;PMML rule is according to the PMML model content of upload Data are handled, the file of upload must be ended up with .pmml, after file uploads successfully, can obtain file name automatically As model name;Kalman filter is filtered using Kalman to data are flowed into.One module represents, one rule, than If a drools file is exactly a module, this module is exactly the rule described in this drools file.

Serializing operation includes Kafka Avro serializing and Kafka Arvo unserializing, and Kafka Avro serializing will Java Map turns to binary data according to Avro schema sequence, Kafka Arvo unserializing by binary data according to Avro schema is deserialized as Java Map.

Statistics includes that uniqueness counts, frequency statistics and quantile are estimated;Uniqueness is counted based on field to inflow data The calculating of value uniqueness is carried out, configuration needs the field name counted, carries out statistics frequency of occurrence according to field name, then configuration statistics The setting (configuring newly-generated data to be stored in which column, i.e. in which key) that value field name is exported;Frequency statistics Imprecise computation is executed in a stream, by configuring the field name to be counted, after statistics, is saved in statistical value field name In;Quantile estimation carries out quantile estimation to data are flowed into based on field and specified fractile, configures the field to be estimated Title is estimated according to fractile, finally using the quantile field name of configuration as field name when output.I.e. configuration is new Which column is the data of generation be stored in, i.e. in which key.

Stream operation includes joint, Field-Partitions, value subregion, UDF (user-defined function, user-defined Function) operator and complicated event matching;Combining is to be joined to a upstream, if there is multiple upstream nodes, by upstream Output data joins together to be transmitted to downstream again;Field-Partitions are based on certain field and carry out attitude layer, configure option field, Divisional type is set, subregion is carried out to data according to divisional type and field;It is poly- according to field attribute value progress subregion to be worth subregion It closes, usually docks window class operator below, configure optionally field attribute, subregion is carried out to data by field;UDF operator User oneself can define the operator is used to do which operation, if having selected filter type, the class filled in action name Filter class or its subclass must be inherited from;Complicated event matching carries out matching treatment according to match pattern condition to upstream events And it returns and meets matched sequence of events set.

Map function includes at least one of the following: Spring rule, window maximum, Attribute Association, obtains city name, item Part filter, unique key filtering, resolver, input format conversion, output format conversion, attribute is promoted, attribute maps, Cache Operation;In Spring rule, if regular design conditions are really to execute the rule operation of configuration, when regular design conditions are set It is defaulted as when being set to * true;Window maximum is that history dominant record will be spilt into downstream, and the field to be compared is arranged, according to Compare field, the dominant record in the same window is spilt into downstream;Attribute Association is associated with difference by relevant field attribute Transaction event, configure major key, field name and field value, field name can be matched with field value, and major key is used to extract field The content of name;It obtains city name and city name is searched by IP, be arranged in input data and represent the field of IP as IP field, city City's field name represents city fields in output data;Conditional filter carries out garbled data by conditional expression, meets item The data of part, which will be retained, flows to next operator;Unique key filtering is filtered data according to key, it is ensured that key is identical Data within a specified time only downstream export one;Resolver is based on value (the value xml/json/csv/ in certain field Regular Expression), parse the value (number of xml/json/csv/Regular Expression format of specific field According to);Output format conversion is to format customization data format and store it in payload, first has to configuration format, selects It selects with the output of which kind of format as a result, mapping entity in addition can be configured, it can be with mapping field title;Input format conversion is by word Symbol string, byte, Map set or other kinds of payload are converted to RT event, and (data is a RT event, is represent The format of data in a stream), a format conversion is carried out to input data, output Schema can be configured for next calculation Son uses, and can also configure mapping entity, and the field that original field is changed to oneself needs (is changed the field name of data, i.e., Column name);Attribute promotion is to promote complex object attribute in Map into top layer Map, fills in the field that promote top layer, root The Map of lower layer is extracted into top layer according to field is filled in;Attribute mapping is to be carried out by SpringEL expression formula to field attribute Map operation, operator can directly operate the column of primary data, and optional includes column to be deleted and column to be transformed, configure wait delete Except column can delete certain column in primary data, column name in primary data can be changed by configuring column to be transformed, be configured to be transformed Column need to fill in calculation expression, it is necessary to be the expression formulas of SpringEL grammer;Cache operation is based on Ignite carry out table wound / reading/writing data is built, grouping key attribute indicates the column for needing to cache, and object class must be then the class for needing the value cached.Window Mouth operation includes window sequence and deleting stale data, and window sequence is to be based on field name in the window temporally to stab sequence, can With option and installment major key, it is polymerize according to major key, is then ranked up further according to the sort key title of configuration;It eliminates expired Data are to be based on timestamp deleting stale data in the window to carry out the time in the window by extracting the time value in data The comparison of stamp eliminates expired data.Sort key seeks to compare sequence with the data in which column, and column name is exactly Here classification key name.

The place point indicates the place of data outflow, may include at least one of following: JDBC for exporting target data Output, Redis output, Kafka output, IBM MQ are exported, based on the Http output of Http operation, the Socket based on data source It exports and the local file based on data source exports.Wherein, Http endpoint is sent data to when Http is exported.JDBC output It is to write data into database, such as MySQL, DB2 with JDBC.Redis output is to write data into Redis.Kafka is exported Transmit data to Kafka.IBM MQ output is to transmit data to IBM MQ.Socket output is to transmit data to Socket.Local file output is to write data into local file.

Referring to FIG. 2, Fig. 2 is the schematic diagram of the first user interface provided in an embodiment of the present invention, first user circle Face is capable of providing the node for creating directed acyclic graph, and the node includes source point, operator and place point.Implementation shown in Fig. 2 In example, first user interface includes the first display area 201 and the second display area 202, and the first display area 201 is used for Show the directed acyclic graph of creation, second display area 202 is for showing the node.

In embodiment shown in Fig. 2, the first user interface can directly provide the second display area 202, for showing section Point, so that user be facilitated to select the node for creating directed acyclic graph.Certainly, in some other embodiment of the invention, Node can not also be directly displayed in first user interface, but the selection of node is provided by other means, for example, providing one Kind node searching tool, as shown in Fig. 2, the search box in the second display area 202, to be searched by inputting in the search box Node keyword, node can also be found.Certainly, other modes for providing node of the first user interface are not limited to above-mentioned Two ways.

In the embodiment of the present invention, in response to the selection operation to the node, the icon of the destination node of selection is shown In in the first display area of the first user interface, the icon is, for example, box shown in Fig. 2.The selection operation can be with For the clicking operation for clicking the node, alternatively, the node to be drawn to the drag operation of first display area.It clicks Operation and the equal very simple of drag operation, and meet user's use habit.

It will include institute using node connecting line in response to the attended operation to the destination node in the embodiment of the present invention State source point, operator is connected with the destination node of place point, is created as directed acyclic graph, the node connecting line is, for example, institute in Fig. 2 The connecting line with the arrow shown.The attended operation can be for from the output anchor point of a destination node to the defeated of another destination node Enter the drag operation of anchor point dragging.In the embodiment of the present invention, in addition to the source point and place point, each operator all has an input Anchor point and an output anchor point;The source point has an output anchor point, realizes connection for the input anchor point with the operator;It is described Place point has an input anchor point, realizes connection for the output anchor point with the operator.When two nodes connect, inputted as data The node of side is known as upstream node, and the node as data output side is known as downstream node.In the embodiment of the present invention, the node Connecting line is formed by drag operation, certainly, can also shape by other means in some other embodiment of the invention At, such as a connecting line drawing tool is provided, node connecting line etc. is drawn by clicking connecting line drawing tool.

Referring to FIG. 2, the directed acyclic graph of creation completion is shown in the first display area 201 in Fig. 2, a creation The directed acyclic graph of the data flow of completion includes source point, at least one operator and place point.In other embodiments, a creation is completed Data flow also may include more than one source point and more than one place point.The section of the directed acyclic graph of display in Fig. 2 In point, local data source is source point, and local file output is place point, other are operator.

In the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the configuration operation to the destination node or the node connecting line in first display area, show Show information configuration interface;

Based on the configuration information that user inputs at the information configuration interface, to the institute shown in first display area It states destination node or the node connecting line is configured.

Referring to FIG. 2, the first user interface in Fig. 2 includes third display area 203, the third display area 203 For showing information configuration interface.The information configuration interface configured to local file source is shown in Fig. 2.It can from Fig. 2 To find out, user can input the configuration information in local file source in the text input box of offer.

In the embodiment of the present invention, the configuration information of the destination node may include at least one of following: title, description letter Breath, output field structure (schema) and parameter configuration.

In the embodiment of the present invention, when carrying out information configuration to node connecting line, it is generally basede on the setting of SpringEL grammer Data select expression formula progress data flow to enter selection.The configuration information of the node connecting line may include at least one of following: Data filtering information and data diffluent information.In the embodiment of the present invention, the node connecting line will be with its input terminal in addition to having The data of the node of connection are directed into except the function for the node connecting with its output end, can also have data filtering and/or The function of data distribution, filtering refer to the data for filtering out certain condition, and shunting refers to the node connecting with its input terminal is defeated Data out are divided at least two parts, are directed respectively into the different nodes connected from its output end.

When carrying out information configuration to source point, can configure at least one of following: the title of source point, description information, output Schema and parameter configuration, local file source can with Profile Path and/or file coding format (generally default UTF-8), The source Socket can configure the address Socket and/or port numbers, and the source Http can configure at least one of following: obtain data when Between be spaced (necessarily), http endpoint network address URL, http Method type, receive data format (xml, json, form), The source JDBC can configure at least one of following: driving title, database connection network address, user name, password, for loading data SQL, analog data source generally need setup time interval and reading value, and IBM MQ can configure at least one of following: host address, Target port, channel, user id, password, target CCSID, queue management device, queue or subject name, maximum reconnect number, The source Kafka can configure at least one of following: subscribe to subject, the address kafka, consumption group mark, deviation post of data.

When carrying out information configuration to operator, can configure at least one of following: the title of operator, description information, output Schema and parameter configuration, its parameter configuration of different operators are also different.It is represented due to different operators at different data Logic is managed, therefore the configuration of different operators is also not quite similar.Such as window Aggregation Operator, configuration is divided into two parts: a part is window Relevant configuration, how configuration information is for by data to carry out classified packets, as shown in Figure 3.In Fig. 3, it is one section that major key, which generates class, Code represents the logic for generating major key, can be customized by the user.Major key and major key, which generate class, one, two choosings One.Global window is the window classified without major key, and all data belong to same group.Window type refers to different windows Mouthful, there are tumbling type and slidling type.Window size, that is, window duration.A part is matched for polymerization correlation It sets, configuration information has indicated how by the data progress converging operation after classified packets, as shown in Figure 4.Grouping is data point It opens, polymerization is the later data of statistical packet.Such as batch of data 1,2,3,4,5,6 is divided into 2 groups according to certain rule, It is respectively as follows: 1,2,3 and 4,5,6, then does the converging operation of their accumulated values, then first group of result is 6, second group of knot Fruit is 15.Polymeric type is just different converging operation, such as accumulated value, maximum value, minimum value, average value, variance etc..It is poly- Closing attribute is to carry out the converging operation that front is said to the value in which field.Adeditive attribute is certain converging operation needs Other attributes, such as the converging operation of TopN need specified N is how many.

When clicking through row information configuration to place, can configure at least one of following: title, the description information, output of place point Schema and parameter configuration, the parameter type of homoclinic point configuration is not different.As local file output can with Profile Path and/ Or file write mode (newly-built or covering), Http output can configure at least one of following: Thread Count, maximum thread, block Queue size, URL, http method and reception data format.

In the embodiment of the present invention, between node and node and it is not all can connect, such as the data of source point can only flow out, The data of place point can only flow into, and the data of operator can flow in and out, but cannot form loop, etc., node and node Between can connect, referred to as nodes match cannot connect between node and node, and referred to as two nodes mismatch.The present invention is real It applies in example, the matching rule of these nodes can be saved.In order to remind user, in the embodiment of the present invention, the destination node packet Output anchor point and/or input anchor point are included, it is described in response to the target section in response to the attended operation to the destination node The attended operation of point will be connected with the destination node of place point including the source point, operator using node connecting line and include:

It is shown based on default display mode and exports the matched input anchor point of anchor point with target;

Target input anchor point is connected to from target output anchor point using node connecting line.

The default display mode of institute, such as can be to be highlighted and export the matched input anchor point of anchor point with target, alternatively, will With the matched input anchor point amplification display of target output anchor point, etc., if reach differentiation and target to export anchor point matched defeated Enter anchor point and the purpose with the unmatched input anchor point of target output anchor point.

In above-described embodiment, the creation of directed acyclic graph can be completed, it, can also be to the first user circle after the completion of creation Directed acyclic graph on face is edited, it is described editor may include: preservation, publication, importing, export, editor application message, Jar assures reason, automatic placement, multiple selection modes, duplication are pasted, the first user interface of resetting, in shortcut key at least one of.

1, save and issue directed acyclic graph

In the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the preservation operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code saves.

Referring to FIG. 2, the first user interface in Fig. 2 can also provide one the 4th display area 204, the 4th display A preservation icon can be shown in region 204, saved icon by clicking, can be saved the directed acyclic graph of creation.When with When preservation icon is clicked at family, directed acyclic graph created can be converted into the code of json format, include stream in the code Using (real-time streams run in RT are defined by application one by one.Each application is stream application) in each node match Confidence breath and upstream and downstream information, such as pull a node it is carried out up and down line configuration when, the upstream ID config option of the node In preserve upstream node ID, similarly, the downstream ID of the node also can be set, if the upstream node ID of the node be 1, under Swimming node ID is 1, then the ID of the node itself is 2, and the upstream ID corresponds to the input anchor point of the node, and the downstream ID is corresponding The output anchor point of the node.When as saved icon when the user clicks, the directed acyclic graph in the first user interface can be turned from the background The code for being changed to json format is saved to database.

In the embodiment of the present invention, directed acyclic graph is converted into the code of json format, json (JavaScript Object Notation, JS object numbered musical notation) be a kind of lightweight data interchange format.It is by ECMAScript (based on Europe Suan Ji association formulate js specification) a subset, stored and indicated using the text formatting for being totally independent of programming language Data.Succinctly and clearly hierarchical structure makes json become ideal data interchange language.It is easy to people to read and write, simultaneously It is also easy to machine parsing and generates, and effectively promote network transmission efficiency.

In the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the publication operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code;

It is the corresponding data flow of Flink by the code conversion of the json format;

The data flow is run, stream operation is generated.

Referring to FIG. 2, a publication icon can be shown in the 4th display area 204, icon is issued by clicking, The directed acyclic graph of creation can be issued.Referring to FIG. 2, when issuing icon when the user clicks, by having in the first user interface The code json code for being converted to json format to acyclic figure is further converted into the corresponding data flow of Flink, and automatic running. To realize the processing to data, in concrete processing procedure, the data that the node connecting line obtains the source point are inputted To the operator, and the operator, treated that data are input to the place point to export target data.Flink is a point The computing engines of cloth, it can be used to do stream process, i.e., handle some real-time streams in real time, generate data in real time As a result.

The data flow is, for example, flow diagram (JobGraph), and JobGraph is the unique data flow engine institute by Flink The data structure of the statement operation of identification.

2, the importing and export of directed acyclic graph

Icon and export icon are imported (after issuing referring to FIG. 2, can show in the 4th display area 204 Two icons), by click import icon and export icon, can import icon and export icon directed acyclic graph.

That is, in the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the export operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code, and be compressed file (such as ZIP file) export by the code conversion of the json format.

Transmission etc. so as to realize directed acyclic graph operates.Derived file can be used for newly-built complete application.

In the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the import operation of the directed acyclic graph to creation, the directed acyclic graph after file is converted will be imported and be shown in Current interface.

Further, the import operation in response to the directed acyclic graph to creation will import having after file is converted Being shown in current interface to acyclic figure can also include:

In response to importing the import operation of the code of the json format of compression, the generation of the json format of the compression is decompressed Code;

It is directed acyclic graph by the code conversion of the json format after decompression, and covers current in first display area The directed acyclic graph of creation.

Existing directed acyclic graph is imported into system so as to realize.

In the embodiment of the present invention, is imported and exported with json code.Export is usually to be converted to directed acyclic graph Json code is simultaneously further compressed into compressed file and exports to database or local.Import be usually upload from RT application derived from Compressed file, the stream application that covering is currently editing by import operation, and can not restore.

3, application message is edited

First user interface further includes editor's application message icon, clicks editor's application message icon, will pop up one Messagewindow is edited, as shown in Figure 5.

Wherein, timeliness is determined by the data in stream application, each data corresponding time, if data itself are wrapped It then handles, can be handled according to consumption time if itself does not include the time, when consumption according to event time containing event time Between be exactly time that this data enters in system, runing time is the time that this data is handled by system.Watermark interval is The attribute of Flink itself, there is window converging operation in flink, window according to watermark interval determine window how cutting or every Which window data belongs to.

Checkpoint configuration is the attribute configuration for adjusting stream application operation and flink itself, there is default value, user It can also custom-configure.

Restart the attribute configuration that strategy configuration is also flink itself, has default value, user can also custom-configure.

4, Jar assures reason

As shown in Fig. 2, first user interface can also include the icon (one i.e. after setting icon that Jar assures reason A icon), it clicks Jar and assures reason icon, will pop up Jar and assure reason window, as shown in Figure 6.By interface user can more than Pass jar file, mainly operator.

That is, in the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to uploading the upload operation of operator, the operator of upload is added to the node for being used to create directed acyclic graph In.

That is, except the operator for having deposited system offer, user can also upload other calculations in the embodiment of the present invention Son, to improve user experience.

5, automatic placement

First user interface further includes the icon of automatic placement, automatic placement icon is clicked, to the first user interface In mixed and disorderly node and node connecting line carry out automatic placement, help user to set up the directed acyclic of a beauty and specification Figure.

6, multiple selection modes

As shown in Fig. 2, first user interface further includes icon (the 8th figure i.e. after publication icon of multiple selection modes Mark), multiple selection modes icon is clicked, multiple nodes in the first user interface can be selected, with choice box in order to replicate, move Or delete choice.

7, duplication is pasted

As shown in Fig. 2, first user interface further includes icon (the 9th figure i.e. after publication icon that duplication is pasted Mark), duplication paste icon is clicked, can choose some or all of node and node connecting line in directed acyclic graph, is replicated To other directed acyclic graph, with all information of node and node connecting line, including configuration information when duplication.

8, the first user interface is reset

First user interface can also include the icon of the first user interface of resetting, for the first user interface to contract Ratio is put, at the same time it can also which directed acyclic graph to be shown centered on.

9, shortcut key

First user interface can also include shortcut key, including amplification, reduce, reform, recalling, replicating and/or Prompt operation is supported in stickup etc., provides night mode, convenient to be shown in projection or large-size screen monitors.

In the embodiment of the present invention, after having created directed acyclic graph, directed acyclic graph can also be debugged, be checked Log and/or monitoring.

1, it debugs

Directed acyclic graph creation complete after, can stream operation corresponding to the directed acyclic graph of creation debug, In order to developer debugging window in be currently running stream operation input test data, observe output result whether Meet expection, to pass through the modification stream operation of the configuration information of update operator.

Debugging window can be as shown in FIG. 7 and 8.

I.e. in the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to the debugging operations debugged to the stream operation, the stream operation is debugged, and show debugging As a result.

2, log is checked

After directed acyclic graph creation is completed, the log of the corresponding stream operation of directed acyclic graph can be checked, in order to Exploitation or operation maintenance personnel pass through the real-time running state for checking the log observation stream operation exported in operation in stream operation.Such as In Risk Monitoring, risk information can be known in real time.

That is, in the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

In response to checking operation to the log of the stream operation, the log of the stream operation is shown.

3, it monitors

After directed acyclic graph creation is completed, it can be monitored with the treatment process of each operator in convection current operation, In order to develop or operation maintenance personnel stream operation in by check flink monitoring information observe stream operation real time execution shape State.As shown in Figure 9.

I.e. in the embodiment of the present invention, the data processing method based on directed acyclic graph can also include:

Operation is checked in response to the operating status to the stream operation, is obtained in the stream operation at least partly described The monitoring information of node, and show the monitoring information.

Based on the same inventive concept, referring to FIG. 10, the embodiment of the present invention also provides a kind of number based on directed acyclic graph According to processing system, comprising:

Display module 101, for showing the first user interface, first user interface includes for creating directed acyclic The node of figure, the node include source point for obtaining the input data of data source, for being handled input data Operator and place point for exporting target data;

First execution module 102, in response to the selection operation to the node, by the icon of the destination node of selection It is shown in the first display area of the first user interface;

Second execution module 103, for that will be wrapped using node connecting line in response to the attended operation to the destination node Include the source point, operator is connected with the destination node of place point, be created as directed acyclic graph;Wherein, in the directed acyclic graph, The data that the source point obtains are input to the operator by the node connecting line, and treated that data input the operator To the place point to export target data.

Optionally, first user interface further includes the second display area, and second display area is for showing institute State node.

Optionally, the selection operation is to click the clicking operation of the node, alternatively, the node is drawn to described The drag operation of first display area.

Optionally, the destination node includes output anchor point and/or input anchor point, and the attended operation is from a mesh Mark the drag operation that the output anchor point of node is pulled to the input anchor point of another destination node.

Optionally, the system also includes:

Third execution module, in response to in first display area the destination node or the node connect The configuration of wiring operates, and shows information configuration interface;

4th execution module, the configuration information for being inputted based on user at the information configuration interface, to described first The destination node or the node connecting line shown in display area is configured.

Optionally, the configuration information of the destination node includes at least one of the following: title, description information, output field Structure and parameter configuration.

Optionally, the configuration information of the node connecting line includes at least one of the following: data filtering information and data point Stream information.

Optionally, the destination node includes that output anchor point and/or input anchor point, second execution module are also used to It is shown based on default display mode and exports the matched input anchor point of anchor point with target;It is exported using node connecting line from the target Anchor point is connected to target input anchor point.

Optionally, the system also includes:

5th execution module, for the preservation operation in response to the directed acyclic graph to creation, by the directed acyclic of creation The code that figure is converted into json format saves.

Optionally, the system also includes:

6th execution module, for the publication operation in response to the directed acyclic graph to creation, by the directed acyclic of creation Figure is converted into the code of json format;It is the corresponding data flow of Flink by the code conversion of the json format;Run the number According to stream, stream operation is generated.

Optionally, the system also includes:

7th execution module, for the export operation in response to the directed acyclic graph to creation, by the directed acyclic of creation Figure is converted into the code of json format, and is compressed file export by the code conversion of the json format.

Optionally, the system also includes:

8th execution module, for the import operation in response to the directed acyclic graph to creation, after file conversion will be imported Directed acyclic graph be shown in current interface.

Optionally, the system also includes:

8th execution module is further used for the import operation of the code of the json format in response to importing compression, Decompress the code of the json format of the compression;It is directed acyclic graph by the code conversion of the json format after decompression, and covers The directed acyclic graph currently created.

Optionally, the system also includes:

The operator of upload is added to for the upload operation in response to uploading operator and is used to create by the 9th execution module In the node of directed acyclic graph.

Optionally, the system also includes:

Tenth execution module, in response to the debugging operations debugged of stream operation, to the stream operation into Row debugging, and show debugging result.

Optionally, the system also includes:

11st execution module, for showing the stream operation in response to checking operation to the log of the stream operation Log.

Optionally, the system also includes:

12nd execution module obtains the stream for checking operation in response to the operating status to the stream operation To the monitoring information of at least partly described node in operation, and show the monitoring information.

Figure 11 is please referred to, Figure 11 is the structural schematic diagram of the processing equipment of one embodiment of the invention, the processing equipment 110 packet It includes: processor 111 and memory 112.In embodiments of the present invention, processing equipment 110 further include: be stored on memory 112 And the computer program that can be run on processor 111, computer program realize following steps when being executed by processor 111:

Show that the first user interface, first user interface include the node for creating directed acyclic graph, the section Point includes the source point for obtaining the input data of data source, the operator for being handled input data and for exporting mesh Mark the place point of data;

In response to the selection operation to the node, the icon of the destination node of selection is shown in the first user interface In first display area;

It will include the source point, operator and place using node connecting line in response to the attended operation to the destination node The destination node connection of point, is created as directed acyclic graph;

Wherein, in the directed acyclic graph, the data that the source point obtains are input to described by the node connecting line Operator, and treated that data are input to the place point to export target data the operator.It is total that processor 111 is responsible for management Line architecture and common processing, memory 112 can store the used data when executing operation of processor 111.

Optionally, first user interface further includes the second display area, and second display area is for showing institute State node.

Optionally, the attended operation is from the output anchor point of a destination node to the defeated of another destination node Enter the drag operation of anchor point dragging.

Optionally, the destination node includes output anchor point and/or input anchor point, and the attended operation is from a mesh The drag operation that the anchor point of mark node is pulled to the anchor point of another destination node.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the configuration operation to the destination node or the node connecting line in first display area, show Show information configuration interface;

Based on the configuration information that user inputs at the information configuration interface, to the institute shown in first display area It states destination node or the node connecting line is configured.

Optionally, the configuration information of the destination node includes at least one of the following: title, description information, output field Structure and parameter configuration.

Optionally, the configuration information of the node connecting line includes at least one of the following: data filtering information and data point Stream information.

Optionally, the destination node includes that output anchor point and/or input anchor point, computer program are held by processor 111 It can also be achieved following steps when row:

It is shown based on default display mode and exports the matched input anchor point of anchor point with target;

Target input anchor point is connected to from target output anchor point using node connecting line.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the preservation operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code saves.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the publication operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code;

It is the corresponding data flow of Flink by the code conversion of the json format;

The data flow is run, stream operation is generated.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the export operation of the directed acyclic graph to creation, the directed acyclic graph of creation is converted into json format Code, and be compressed file export by the code conversion of the json format.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the import operation of the directed acyclic graph to creation, the directed acyclic graph after file is converted will be imported and be shown in Current interface.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to importing the import operation of the code of the json format of compression, the generation of the json format of the compression is decompressed Code;

It is directed acyclic graph by the code conversion of the json format after decompression, and covers current in first display area The directed acyclic graph of creation.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to uploading the upload operation of operator, the operator of upload is added to the node for being used to create directed acyclic graph In.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to the debugging operations debugged to the stream operation, the stream operation is debugged, and show debugging As a result.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

In response to checking operation to the log of the stream operation, the log of the stream operation is shown.

Optionally, following steps be can also be achieved when computer program is executed by processor 111:

Operation is checked in response to the operating status to the stream operation, is obtained in the stream operation at least partly described The monitoring information of node, and show the monitoring information.

The embodiment of the present invention also provides a kind of computer readable storage medium, stores on the computer readable storage medium Computer program, the computer program realize that the above-mentioned data processing method based on directed acyclic graph is real when being executed by processor Each process of example is applied, and identical technical effect can be reached, to avoid repeating, which is not described herein again.Wherein, the calculating Machine readable storage medium storing program for executing, such as read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc..

The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art For, without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications It should be regarded as protection scope of the present invention.

Claims (10)

1. a kind of data processing method based on directed acyclic graph characterized by comprising
Show that the first user interface, first user interface include the node for creating directed acyclic graph, the node packet Include the source point of the input data for obtaining data source, the operator for being handled input data and for exporting number of targets According to place point;
In response to the selection operation to the node, the icon of the destination node of selection is shown in the first of the first user interface In display area;
It will include the source point, operator and place point using node connecting line in response to the attended operation to the destination node Destination node connection, is created as directed acyclic graph;
Wherein, in the directed acyclic graph, the data that the source point obtains are input to the operator by the node connecting line, And treated that data are input to the place point to export target data the operator.
2. the method as described in claim 1, which is characterized in that first user interface further includes the second display area, institute The second display area is stated for showing the node.
3. method according to claim 1 or 2, which is characterized in that the selection operation is to click the click behaviour of the node Make, alternatively, the node to be drawn to the drag operation of first display area.
4. the method as described in claim 1, which is characterized in that the destination node includes exporting anchor point and/or input anchor point, The attended operation is to drag from the output anchor point of a destination node to what the input anchor point of another destination node pulled Drag operation.
5. the method as described in claim 1, which is characterized in that further include:
In response to the configuration operation to the destination node or the node connecting line in first display area, display letter Cease configuration interface;
Based on the configuration information that user inputs at the information configuration interface, to the mesh shown in first display area Mark node or the node connecting line are configured.
6. a kind of data processing system based on directed acyclic graph characterized by comprising
Display module, for showing the first user interface, first user interface includes the section for creating directed acyclic graph Point, the node include the source point for obtaining the input data of data source, the operator for being handled input data and For exporting the place point of target data;
First execution module, in response to the selection operation to the node, the icon of the destination node of selection to be shown in In first display area of the first user interface;
Second execution module, for that will include described using node connecting line in response to the attended operation to the destination node Source point, operator are connected with the destination node of place point, are created as directed acyclic graph;Wherein, in the directed acyclic graph, the section The data that the source point obtains are input to the operator by point connecting line, and treated described in data are input to the operator Place point is to export target data.
7. system as claimed in claim 6, which is characterized in that first user interface further includes the second display area, institute The second display area is stated for showing the node.
8. system as claimed in claims 6 or 7, which is characterized in that the selection operation is to click the click behaviour of the node Make, alternatively, the node to be drawn to the drag operation of first display area.
9. system as claimed in claim 6, which is characterized in that the destination node includes exporting anchor point and/or input anchor point, The attended operation is to drag from the output anchor point of a destination node to what the input anchor point of another destination node pulled Drag operation.
10. system as claimed in claim 6, which is characterized in that further include:
Third execution module, in response to the destination node or the node connecting line in first display area Configuration operation, show information configuration interface;
4th execution module, the configuration information for being inputted based on user at the information configuration interface, to first display The destination node or the node connecting line shown in region is configured.
CN201811644309.9A 2018-12-29 2018-12-29 A kind of data processing method and processing system based on directed acyclic graph CN109726216A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811644309.9A CN109726216A (en) 2018-12-29 2018-12-29 A kind of data processing method and processing system based on directed acyclic graph

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811644309.9A CN109726216A (en) 2018-12-29 2018-12-29 A kind of data processing method and processing system based on directed acyclic graph

Publications (1)

Publication Number Publication Date
CN109726216A true CN109726216A (en) 2019-05-07

Family

ID=66299505

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811644309.9A CN109726216A (en) 2018-12-29 2018-12-29 A kind of data processing method and processing system based on directed acyclic graph

Country Status (1)

Country Link
CN (1) CN109726216A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111506779A (en) * 2020-04-20 2020-08-07 东云睿连(武汉)计算技术有限公司 Object version and associated information management method and system facing data processing

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101859313A (en) * 2009-04-08 2010-10-13 索尼公司 Messaging device and method and program thereof
CN102508706A (en) * 2011-11-18 2012-06-20 北京航空航天大学 Multi-source data integrating platform and establishing method thereof
CN103677763A (en) * 2012-08-30 2014-03-26 中国科学院软件研究所 Graphical programming source file storage and analytic method
CN105303397A (en) * 2015-09-28 2016-02-03 百度在线网络技术(北京)有限公司 Information recommendation method, system, server end and client
CN105843630A (en) * 2016-06-08 2016-08-10 江西洪都航空工业集团有限责任公司 Method for graphical programming development based on robot
US20170017903A1 (en) * 2015-02-11 2017-01-19 Skytree, Inc. User Interface for a Unified Data Science Platform Including Management of Models, Experiments, Data Sets, Projects, Actions, Reports and Features
CN108830383A (en) * 2018-05-30 2018-11-16 第四范式(北京)技术有限公司 For showing the method and system of machine learning model building process
CN108898229A (en) * 2018-06-26 2018-11-27 第四范式(北京)技术有限公司 For constructing the method and system of machine learning modeling process
CN109033109A (en) * 2017-06-09 2018-12-18 杭州海康威视数字技术股份有限公司 Data processing method and system

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101859313A (en) * 2009-04-08 2010-10-13 索尼公司 Messaging device and method and program thereof
CN102508706A (en) * 2011-11-18 2012-06-20 北京航空航天大学 Multi-source data integrating platform and establishing method thereof
CN103677763A (en) * 2012-08-30 2014-03-26 中国科学院软件研究所 Graphical programming source file storage and analytic method
US20170017903A1 (en) * 2015-02-11 2017-01-19 Skytree, Inc. User Interface for a Unified Data Science Platform Including Management of Models, Experiments, Data Sets, Projects, Actions, Reports and Features
CN105303397A (en) * 2015-09-28 2016-02-03 百度在线网络技术(北京)有限公司 Information recommendation method, system, server end and client
CN105843630A (en) * 2016-06-08 2016-08-10 江西洪都航空工业集团有限责任公司 Method for graphical programming development based on robot
CN109033109A (en) * 2017-06-09 2018-12-18 杭州海康威视数字技术股份有限公司 Data processing method and system
CN108830383A (en) * 2018-05-30 2018-11-16 第四范式(北京)技术有限公司 For showing the method and system of machine learning model building process
CN108898229A (en) * 2018-06-26 2018-11-27 第四范式(北京)技术有限公司 For constructing the method and system of machine learning modeling process

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111506779A (en) * 2020-04-20 2020-08-07 东云睿连(武汉)计算技术有限公司 Object version and associated information management method and system facing data processing

Similar Documents

Publication Publication Date Title
US10778712B2 (en) Displaying network security events and investigation activities across investigation timelines
US10678805B2 (en) Schedule modification of data collection requests sent to external data sources
US20200097466A1 (en) Method and system for implementing target model configuration metadata for a log analytics system
US10848510B2 (en) Selecting network security event investigation timelines in a workflow environment
US9886469B2 (en) System performance logging of complex remote query processor query operations
US10324761B2 (en) Providing configurable workflow capabilities
US20170142143A1 (en) Identifying notable events based on execution of correlation searches
US9800675B2 (en) Methods for dynamically generating an application interface for a modeled entity and devices thereof
US20200201850A1 (en) In-Memory Catalog for Searching Metrics Data
CN104854583B (en) Search result rank and presentation
US20180081935A1 (en) Data Visualization in a Dashboard Display Using Panel Templates
US20190306184A1 (en) Detection of anomalies in a time series using values of a different time series
US20150286677A1 (en) Analyzing event streams of user sessions
US10061807B2 (en) Collection query driven generation of inverted index for raw machine data
EP2616965B1 (en) Support for a parameterized query/view in complex event processing
US20170052977A1 (en) Apparatus and Method for Collaboratively Analyzing Data Snapshot Visualizations from Disparate Data Sources
CN102349056B (en) Dynamically composing data stream processing applications
KR101644418B1 (en) Visualizing relationships between data elements and graphical representations of data element attributes
US20170031565A1 (en) Network security investigation workflow logging
US8429527B1 (en) Complex data merging, such as in a workflow application
US7840604B2 (en) Method, apparatus and computer program for managing the processing of extracted data
US8701080B2 (en) Template components having constraints representative of best practices in integration software development
US7716253B2 (en) Centralized KPI framework systems and methods
US7284245B2 (en) Query trees including or nodes for event filtering
US8095507B2 (en) Automated topology-based statistics monitoring and performance analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination