CN112559808A - Data processing method and device and electronic equipment - Google Patents
Data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN112559808A CN112559808A CN202011497745.5A CN202011497745A CN112559808A CN 112559808 A CN112559808 A CN 112559808A CN 202011497745 A CN202011497745 A CN 202011497745A CN 112559808 A CN112559808 A CN 112559808A
- Authority
- CN
- China
- Prior art keywords
- rule
- data
- processing
- input data
- instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The disclosure relates to a data processing method and device and electronic equipment, and relates to the technical field of cloud computing and internet of things. The specific implementation scheme is as follows: acquiring input data; and processing the input data by adopting a pre-configured rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic, so that the input data is processed by the at least two data processing rules, and the data processing efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to the field of cloud computing and internet of things technologies. Specifically, a data processing method, a data processing device and electronic equipment are provided.
Background
The rule engine system is generally applied to data intensive scenes, particularly to scenes with rapid change of services or businesses, and under the scene that related business data systems need to be updated rapidly and at low cost, the rule engine can better meet various requirements of businesses on new scenes, data conversion of different data formats, data filtering, data stream conversion and the like. Especially in the scene of the internet of things, thousands of devices developed by different manufacturers may be contained in a mass of devices, the data formats of different devices or devices of different manufacturers are different, the rule engine system is very suitable for the scene, and a service manager only needs to configure different rules to complete data processing of different application scenes.
Disclosure of Invention
The disclosure provides a data processing method and device and electronic equipment.
According to a first aspect of the present disclosure, there is provided a data processing method, including:
acquiring input data;
and processing the input data by adopting a preset rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic.
According to a second aspect of the present disclosure, there is provided a data processing apparatus comprising:
the acquisition module is used for acquiring input data;
and the processing module is used for processing the input data by adopting a preset rule chain to obtain target data, and the rule chain is formed by at least two data processing rules according to topological logic.
According to a third aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the first aspects.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of the first aspects.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of a data processing method provided by an embodiment of the present disclosure;
FIG. 2 is another flow chart of a data processing method provided by an embodiment of the present disclosure;
fig. 2a is a relationship diagram among a rule management module, a data access module, and a rule execution module provided in the embodiment of the present disclosure;
FIG. 2b is a schematic diagram of an example of a rule run provided by an embodiment of the present disclosure;
fig. 2c is a flowchart of a data processing method provided by the embodiment of the present disclosure;
FIG. 2d is a further flowchart of a data processing method provided by the embodiments of the present disclosure;
fig. 2e is a schematic diagram of a data access module provided by an embodiment of the present disclosure;
FIG. 2f is a diagram of a relationship between a rule execution module and a data access module provided by an embodiment of the present disclosure;
FIG. 3 is a block diagram of a data processing apparatus provided by an embodiment of the present disclosure;
fig. 4 is a block diagram of an electronic device for implementing a data processing method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Referring to fig. 1, fig. 1 is a flowchart of a data processing method provided in an embodiment of the present disclosure, and as shown in fig. 1, the embodiment provides a data processing method applied to an electronic device, including the following steps:
And 102, processing the input data by adopting a preset rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic.
The rule chain is configured in advance, during configuration, at least two data processing rules can be set, then the execution sequence of each data processing rule in the at least two data processing rules is set, namely, each data processing rule is connected and configured according to topology logic to form a rule chain, and the data processing rules in the rule chain can comprise a data filtering rule and a data conversion rule.
The electronic device processes the input data according to the rule set of the rule chain to obtain the target data, where the target data may have a preset data format, for example, the data format refers to the number of parameters included in the target data and the data type of the parameters. The data format of the target data can be configured according to the rule chain, namely different data formats of the target data can be set by configuring different rule chains, so that the flexibility of acquiring the target data can be improved, and more scene requirements for acquiring different target data can be met
In the embodiment, input data is acquired; and processing the input data by adopting a pre-configured rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic, so that the input data is processed by the at least two data processing rules, and the data processing efficiency is improved.
Referring to fig. 2, fig. 2 is a flowchart of a data processing method provided in an embodiment of the present disclosure, and as shown in fig. 2, the embodiment provides a data processing method applied to an electronic device, including the following steps:
the method comprises the steps of requesting for obtaining through a hypertext transfer protocol, obtaining from a subscribed Message queue, obtaining through reading a preset database, obtaining from a subscribed Message queue Telemetry transmission Message agent (namely, MQTT Broker, wherein the MQTT is a Message queue Telemetry transmission protocol (Message queue Telemetry Transport)) a preset theme (Topic), and obtaining through reading a preset file.
In the above, the preset database includes, but is Not limited to, a relational database, a non-relational database (Not Only SQL, NoSQL), such as Redis, MongoDB, and the like. The preset database may also be a designated table in a user-configured database of Mysql (i.e., a relational database management system), and the input data is data in the designated table.
The preset File includes, but is not limited to, an FTP File (FTP is File Transfer Protocol) and an object storage File. For example, the user may configure an FTP file address, and obtain the input data through the FTP file address.
The method for processing the data can specifically set the input data acquisition mode according to actual requirements, and acquire the input data in different modes, so that the data processing method can process the data in more scenes, and the adaptability of data processing is improved. The input data can be understood as data to be processed.
Further, the input data may also be obtained based on user-configured input rules, such as input rules configured to request input data via hypertext transfer protocol, and/or input rules configured to obtain input data from a subscribed message queue. There may be only one run instance of the input rule, or there may be at least two run instances, for example, as shown in fig. 2e, there are 2 run instances of the rule requesting to obtain input data through the hypertext transfer protocol, and there are 2 run instances of the rule requesting to obtain input data from the subscribed message queue.
For each data processing rule in the rule chain, a corresponding rule run instance needs to be generated to process the input data. For example, if the rule chain includes a data filtering rule and a data conversion rule that are obtained by sorting according to the topological sequence, the rule running instance of the data filtering rule and the rule running instance of the data conversion rule are required to be used to process the input data according to the topological sequence. That is to say, the input data is processed by adopting the rule chain running examples corresponding to the rule chain, the rule chain running examples consist of the rule running examples corresponding to the data processing rules in the rule chain, and the rule running examples process the input data according to the sequence of the topological sequence of the rule chain.
For each first data processing rule of the at least two data processing rules, there may be a plurality of rule running instances in the electronic device, which may be all idle, may be partially in use, or may be all in use. In this embodiment, one of the idle first rule running instances needs to be selected to process the input data. For each first data processing rule, an idle first rule running instance is obtained first, and then the first rule running instance is adopted to participate in the processing of input data.
In the foregoing, the selected first rule running instance is used to process the input data, and is not limited to that the first rule running instance directly processes the input data, or that the first rule running instance indirectly processes the input data, for example, the first rule running instance may process data output by a rule running instance that is ordered before the first data processing rule according to topology in the rule chain. That is, the input data is processed by using the selected first rule running instance, which is understood to be participating in the input data processing.
Step 202 is a specific implementation of step 102.
In this embodiment, input data is acquired; for each first data processing rule in the at least two data processing rules, if the electronic device includes at least one idle first rule running instance of the first data processing rule, selecting one idle first rule running instance from the at least one idle first rule running instance to process the input data, so as to obtain target data and improve data processing efficiency.
Further, if the electronic device does not include at least one idle first rule running instance of the first data processing rule, waiting until there is an idle first rule running instance. Or, generating more first rule running instances, that is, the embodiment of the present disclosure further provides another specific implementation manner of step 102, including the following two steps:
for each first data processing rule in the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generating at least one first rule running instance according to the first data processing rule;
and selecting a first rule running example from the at least one first rule running example to process the input data to obtain target data.
For each data processing rule in the rule chain, a corresponding rule run instance needs to be generated to process the input data. For example, if the rule chain includes a data filtering rule and a data conversion rule that are obtained by sorting according to the topological sequence, the rule running instance of the data filtering rule and the rule running instance of the data conversion rule are required to be used to process the input data according to the topological sequence.
For each first data processing rule of the at least two data processing rules, there may be a plurality of rule running instances in the electronic device, which may be all idle, may be partially in use, or may be all in use. When the input data is processed, one of the idle first rule operation instances needs to be selected to process the input data. For each first data processing rule, an idle first rule running instance is obtained first, and then the first rule running instance is adopted to participate in the processing of input data.
For each first data processing rule, if the electronic device does not include an idle rule running instance of the first data processing rule, generating at least one first rule running instance according to the first data processing rule. It is worth mentioning that the sum of the generated number of the at least one first rule running instance and the existing number of first rule running instances cannot exceed the total number of preconfigured first rule running instances.
In the foregoing, the selected first rule running instance is used to process the input data, and is not limited to that the first rule running instance directly processes the input data, or that the first rule running instance indirectly processes the input data, for example, the first rule running instance may process data output by a rule running instance that is ordered before the first data processing rule according to topology in the rule chain. That is, the input data is processed by using the selected first rule running instance, which is understood to be participating in the input data processing.
Further, step 102 further includes a specific implementation manner, which specifically includes the following three steps: for each first data processing rule in the at least two data processing rules, if the electronic device includes at least one idle first rule running instance of the first data processing rule, selecting one idle first rule running instance from the at least one idle first rule running instance;
for each second data processing rule in the at least two data processing rules, if the electronic device does not include an idle rule running instance of the second data processing rule, generating at least one second rule running instance according to the second data processing rule, and selecting one second rule running instance from the at least one second rule running instance;
and processing the input data according to the selected first rule running example and the selected second rule running example to obtain target data.
In the foregoing, for each first data processing rule in the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generating at least one first rule running instance according to the first data processing rule; and selecting the first rule running instance from the at least one first rule running instance to process the input data to obtain target data, so that the efficiency of processing the input data can be improved.
In one embodiment of the present disclosure, the rule chain includes a data filtering rule and a data conversion rule which are obtained by sorting according to a topological sequence;
the selecting a first rule running instance from the at least one first rule running instance to process the input data to obtain target data comprises the following four steps:
for the data filtering rule, selecting one data filtering rule operation instance from the at least one first rule operation instance;
for the data filtering rule, selecting a data conversion rule operation instance from the at least one first rule operation instance;
processing the input data by using the selected data filtering rule operation example to obtain a first processing result;
and if the first processing result is a preset result, processing the input data by using the selected data conversion rule operation example to obtain the target data.
In the embodiment, the rule chain includes a data filtering rule and a data conversion rule, and in the embodiment, a data filtering rule running instance and a data conversion rule running instance corresponding to the data filtering rule and the data conversion rule respectively are obtained first.
For the data filtering rules, the at least one first rule running instance is at least one data filtering rule running instance, and one data filtering rule running instance is selected from the at least one data filtering rule running instance to participate in the subsequent processing of the input data; for the data conversion rule, the at least one first rule run instance is at least one data conversion rule run instance, and one data conversion rule run instance is selected from the at least one data conversion rule run instance to participate in subsequent processing of input data.
The data filtering rule can filter out data which do not meet the requirements in the input data; the data conversion rule may process the received data into data meeting requirements, for example, the received data has three attribute values, and one of the attribute values may be selected as output data through the data conversion process.
In the above, the selected data filtering rule running instance is used to process the input data to obtain a first processing result, where the first processing result may be true or false, and if true, the input data is determined to meet the requirement, that is, the input data is data that needs to be further processed and is set by a rule chain; if the result is false, the input data is not qualified, that is, the input data is not the data which needs to be further processed and is set by the rule chain.
The preset result may be set according to an actual situation, for example, may be set to true, and when the first processing result is true, the data conversion rule running instance is used to further process the input data, for example, the received data has three attribute values, and the first attribute value may be selected as the output data, that is, the target data, by the data conversion rule running instance.
In this embodiment, the rule chain includes a data filtering rule and a data conversion rule obtained by sorting according to the topological sequence, and can filter out data that does not meet requirements in the input data, and process the received data into target data that meets requirements, and perform multiple types of processing on the input data at one time, thereby improving the data processing efficiency.
In the above, each data processing rule included in the rule chain may be configured by using JSON domain specific language (hereinafter referred to as JSON DSL), and the following advantages are provided:
the learning cost is low. The data query and processing grammar based on the JSON field specific language is simple, and is convenient for business personnel to learn quickly;
the treatment efficiency is high. The method does not depend on a dynamic language operating environment, and can quickly process a large amount of JSON data through a precompiled expression;
is safe and reliable. Because the method provides related grammar and functions, the method can only be used for processing the data in the JSON format, and does not have the security problem like JavaScript;
the data processing of complex services is supported, and the complex functions can be processed by configuring rules of different functions and linking the rules to form a rule chain.
The following examples of the data processing method provided by the present disclosure are as follows:
as shown in fig. 2a, the data processing method provided by the present disclosure includes a rule management module, a data access module, and a rule execution module.
The user configures each data processing rule in the rule chain through the rule management module, and meanwhile, the running state of each data processing rule can be checked through the rule management module.
After the rule chain is configured, the data access module starts to receive the data source according to the configuration, or reads data from the specified data source, and forwards the received data to the rule operation module.
And after receiving the data, the rule running module loads the rule corresponding to the rule chain, and deploys the rule running instance to different running nodes (which can be randomly deployed to different running nodes) according to the rule configuration. After the rule instance is started, the received data are processed one by one according to the rule configuration.
The rule management module is a service module provided for a user to configure rules, manage rules, and monitor rule operations.
Rule configuration: the user can configure the rules that need to be processed for his business through the interactive interface, including setting rule names, data processing logic (based on JSON DSL syntax), rule running topology, etc.
And (3) regular operation monitoring: the user can also check the running state of the rule through the interactive interface, including the running state of the rule, whether the current rule processes the data abnormally or not, the real-time data processing capability of the current rule, and the like.
And (3) rule management: the user can start and stop the relevant rules through the management platform, and modify, delete and the like the relevant rules.
When the user completes the rule configuration or modification, the related rule configuration information is stored persistently and is marked as a unique version number. After the rule is modified, the rule execution module notifies the relevant rule execution instance in the rule execution module to update the rule.
Rule data processing (based on JSON DSL syntax) sets rules for data processing, can complete filtering and conversion functions on input data, and forwards processed data to subsequent rule processing. It mainly includes two types of rule configurations, namely data filtering rules and data transformation rules. In the setup, the configuration is based on the syntax of JSON DSL.
The JSON DSL syntax is a domain-specific language for processing JSON data. When a user writes data processing logic through the JSON DSL, the rule management module can automatically detect the legality of the grammar of the rule management module. When the rule runs, the relevant rule instance loads the DSL, and lexical analysis and syntactic analysis are sequentially completed on the DSL through a lexical analyzer and a syntactic analyzer, so as to finally generate a corresponding syntactic analysis tree (the DSL of the user rule and the syntactic analysis tree are in one-to-one correspondence). Based on the generated parse tree and the input processing data, the rule run instance will complete the data processing, such as the schematic diagram of the rule run instance shown in fig. 2 b.
To gain a deeper understanding of JSON DSL in the rules, a related brief description of JSON domain specific language syntax follows.
JSON DSL is linguistically understood to be a superset of JSON data types, including string, number, Boolean, null, object, array, function. The main operation symbols are shown in the following table 1:
TABLE 1
The JSON domain specific language has the following functional characteristics:
the method comprises the following steps of fast data query function, fast query supporting array structure data, filtering query supporting array structure data, traversal supporting object attribute, fast generation of array structure data, fast generation of object structure data, Boolean expression supporting, numerous supporting functions for array data types, time related functions supporting, string operation related functions supporting, object data type related operation functions supporting and the like.
In rule configuration, in addition to setting data processing logic, rule topology logic may also be set, that is, connection relationships between data processing rules in a rule chain, rule running instances (for example, the number of rule running instances), and the like may be configured. The user performs connection configuration on different rules, a plurality of rules form a rule chain, and the rule chain can be used for processing data of a complex service scene. Further, the rule instances in the rule chain may be run in instances of different rule run modules.
Fig. 2c is a schematic flow chart of a data processing method, where when data is forwarded from the data access module to the rule running module, the rule running module loads a corresponding rule chain first, and when the data is processed by the "other rule a", the rule chain is sent to the "data filtering rule", and if a calculation result of the "data filtering rule" is true, the data is forwarded to a next rule, otherwise, the data is not forwarded to the next rule. And after the data conversion rule receives the new data, performing conversion processing, and after the processing is finished, sending the processed data to the next rule for processing.
Fig. 2d is a schematic flow chart of another data processing method, in which the service processing logic of the user is complex and the data flow is long. When the data volume is large, and a single rule may have a long processing time due to its complexity, a user may configure multiple rule running instances for the rule to improve the capability of processing data in parallel, thereby improving the data throughput and reducing the data processing delay.
As shown in fig. 2d, the rule C is configured to 3 instances, and the rule B is configured to 2 instances, data sent to the rule C will be processed by any idle instance in the 3 instances, and data sent to the rule C will be processed by any idle instance in the 2 instances.
The data access module completes the receiving/reading and forwarding of the data according to the rules set by the user.
When the rule is configured as a data source type, the rule access module starts a related service according to the configuration, receives/reads data, for example, starts a hypertext Transfer Protocol (HTTP) Protocol, receives data to be processed through the HTTP service, and may also subscribe to a message from a specified message queue (for example, Kafka). After reading the data, the module can forward the read data to the rule operation module.
The data access module is shown in fig. 2e as an example, where rule I starts 2 rule instances, starts an HTTP service, receives data sent through an HTTP interface, and after receiving the received data, sends the received data to a rule configured by a user (i.e., a rule in the rule execution module) for subsequent data processing. Rule J is to subscribe to a message queue to obtain data, and also to configure 2 instances, and after receiving data, it forwards its data to the subsequent configured rule (i.e. the rule in the rule running module) for processing.
The rule running module is a rule running instance management and data calculation module, and generates a corresponding rule instance according to a rule set by a user, and calculates and processes received data. Its data computation is mainly based on JSON DSL syntax.
See fig. 2f for an example of a rule run module and data access module relationship. The data access module loads a data receiving type rule, receives an HTTP request as input data and forwards the HTTP request to a rule B instance running in the rule running module, and the rule B instance receives the data, completes processing and forwards the data to subsequent rule processing according to user configuration.
Referring to fig. 3, fig. 3 is a structural diagram of a data processing apparatus according to an embodiment of the present disclosure, and as shown in fig. 3, the embodiment provides a data processing apparatus 300, executed by an electronic device, including:
an obtaining module 301, configured to obtain input data;
a processing module 302, configured to process the input data by using a preconfigured rule chain to obtain target data, where the rule chain is formed by at least two data processing rules according to a topology logic.
Further, the processing module 302 includes:
the first processing sub-module is configured to, for each first data processing rule of the at least two data processing rules, select one idle first rule running instance from the at least one idle first rule running instance to process the input data if the electronic device includes the at least one idle first rule running instance of the first data processing rule, so as to obtain target data.
Further, the processing module 302 includes:
a generating submodule, configured to, for each first data processing rule of the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generate at least one first rule running instance according to the first data processing rule;
and the second processing submodule is used for selecting one first rule running example from the at least one first rule running example to process the input data to obtain target data.
Further, the rule chain comprises a data filtering rule and a data conversion rule which are obtained by sequencing according to the topological sequence;
the second processing submodule includes:
the first selection unit is used for selecting one data filtering rule running example from the at least one first rule running example for the data filtering rule;
a second selection unit, configured to select, for the data filtering rule, one data conversion rule run instance from the at least one first rule run instance;
the first processing unit is used for processing the input data by using the selected data filtering rule operation example to obtain a first processing result;
and the second processing unit is used for processing the input data by using the selected data conversion rule operation example to obtain the target data if the first processing result is a preset result.
Further, the obtaining module 301 is configured to request to obtain the input data through a hypertext transfer protocol; and/or, acquiring the input data from a subscribed message queue, and/or acquiring the input data by reading a preset database; and/or, acquiring the input data from a preset theme in a subscription message queue telemetry transmission message agent; and/or, acquiring the input data by reading a preset file.
The data processing apparatus 300 provided in the embodiment of the present disclosure can implement each process implemented by the electronic device in the method embodiment of fig. 1 and achieve the same technical effect, and for avoiding repetition, details are not described here again.
The present disclosure also provides an electronic device, a computer product and a readable storage medium according to embodiments of the present disclosure.
As shown in fig. 4, is a block diagram of an electronic device of a data processing method according to an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 4, the electronic apparatus includes: one or more processors 401, memory 402, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 4, one processor 401 is taken as an example.
The memory 402, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the acquisition module 301 and the processing module 302 shown in fig. 3) corresponding to the data processing method in the embodiments of the present disclosure. The processor 401 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 402, that is, implements the data processing method in the above-described method embodiment.
The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the data-processing electronic device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 402 optionally includes memory located remotely from processor 401, which may be connected to data processing electronics over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the data processing method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected by a bus or other means, and fig. 4 illustrates an example of a connection by a bus.
The input device 403 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the data processing electronic apparatus, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 404 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
In the present disclosure, by obtaining input data; and processing the input data by adopting a pre-configured rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic, so that the input data is processed by the at least two data processing rules, and the data processing efficiency is improved.
By acquiring input data; for each first data processing rule in the at least two data processing rules, if the electronic device includes at least one idle first rule running instance of the first data processing rule, selecting one idle first rule running instance from the at least one idle first rule running instance to process the input data, so as to obtain target data and improve data processing efficiency.
For each first data processing rule in the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generating at least one first rule running instance according to the first data processing rule; and selecting the first rule running instance from the at least one first rule running instance to process the input data to obtain target data, so that the efficiency of processing the input data can be improved.
The rule chain comprises a data filtering rule and a data conversion rule which are obtained by sequencing according to the topological sequence, can filter out data which do not meet requirements in input data, can process the received data into target data which meet requirements, and can carry out various processing on the input data at one time, so that the data processing efficiency is improved.
The input data can be acquired through a hypertext transfer protocol request and/or acquired from a subscribed message queue, so that the method disclosed by the invention can process data in more scenes, and the adaptability of data processing is improved.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (13)
1. A data processing method, performed by an electronic device, comprising:
acquiring input data;
and processing the input data by adopting a preset rule chain to obtain target data, wherein the rule chain is formed by at least two data processing rules according to topological logic.
2. The method of claim 1, wherein the processing the input data with a preconfigured rule chain to obtain target data comprises:
for each first data processing rule in the at least two data processing rules, if the electronic device includes at least one idle first rule running instance of the first data processing rule, selecting one idle first rule running instance from the at least one idle first rule running instance to process the input data, so as to obtain target data.
3. The method of claim 1, wherein the processing the input data with a preconfigured rule chain to obtain target data comprises:
for each first data processing rule in the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generating at least one first rule running instance according to the first data processing rule;
and selecting one first rule running example from the at least one first rule running example to process the input data to obtain target data.
4. The method of claim 3, wherein the rule chain comprises data filtering rules and data conversion rules which are obtained by sorting according to topological sequences;
the selecting a first rule running instance from the at least one first rule running instance to process the input data to obtain target data comprises:
for the data filtering rule, selecting one data filtering rule operation instance from the at least one first rule operation instance;
for the data filtering rule, selecting a data conversion rule operation instance from the at least one first rule operation instance;
processing the input data by using the selected data filtering rule operation example to obtain a first processing result;
and if the first processing result is a preset result, processing the input data by using the selected data conversion rule operation example to obtain the target data.
5. The method of claim 1, wherein the obtaining input data comprises:
requesting to acquire the input data through a hypertext transfer protocol;
and/or the presence of a gas in the gas,
acquiring the input data from a subscribed message queue;
and/or the presence of a gas in the gas,
acquiring the input data by reading a preset database;
and/or the presence of a gas in the gas,
acquiring the input data from a preset theme in a subscribed message queue telemetry transmission message agent;
and/or the presence of a gas in the gas,
and acquiring the input data by reading a preset file.
6. A data processing apparatus, executed by an electronic device, comprising:
the acquisition module is used for acquiring input data;
and the processing module is used for processing the input data by adopting a preset rule chain to obtain target data, and the rule chain is formed by at least two data processing rules according to topological logic.
7. The apparatus of claim 6, wherein the processing module comprises:
the first processing sub-module is configured to, for each first data processing rule of the at least two data processing rules, select one idle first rule running instance from the at least one idle first rule running instance to process the input data if the electronic device includes the at least one idle first rule running instance of the first data processing rule, so as to obtain target data.
8. The apparatus of claim 6, wherein the processing module comprises:
a generating submodule, configured to, for each first data processing rule of the at least two data processing rules, if the electronic device does not include an idle rule running instance of the first data processing rule, generate at least one first rule running instance according to the first data processing rule;
and the second processing submodule is used for selecting one first rule running example from the at least one first rule running example to process the input data to obtain target data.
9. The apparatus of claim 8, wherein the rule chain comprises a data filtering rule and a data conversion rule which are obtained by sorting according to a topological sequence;
the second processing submodule includes:
the first selection unit is used for selecting one data filtering rule running example from the at least one first rule running example for the data filtering rule;
a second selection unit, configured to select, for the data filtering rule, one data conversion rule run instance from the at least one first rule run instance;
the first processing unit is used for processing the input data by using the selected data filtering rule operation example to obtain a first processing result;
and the second processing unit is used for processing the input data by using the selected data conversion rule operation example to obtain the target data if the first processing result is a preset result.
10. The apparatus of claim 6, wherein the means for obtaining is configured to:
requesting to acquire the input data through a hypertext transfer protocol;
and/or the presence of a gas in the gas,
acquiring the input data from a subscribed message queue;
and/or the presence of a gas in the gas,
acquiring the input data by reading a preset database;
and/or the presence of a gas in the gas,
acquiring the input data from a preset theme in a subscription message queue telemetry transmission message agent;
and/or the presence of a gas in the gas,
and acquiring the input data by reading a preset file.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-5.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011497745.5A CN112559808B (en) | 2020-12-17 | 2020-12-17 | Data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011497745.5A CN112559808B (en) | 2020-12-17 | 2020-12-17 | Data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112559808A true CN112559808A (en) | 2021-03-26 |
| CN112559808B CN112559808B (en) | 2023-07-25 |
Family
ID=75064671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011497745.5A Active CN112559808B (en) | 2020-12-17 | 2020-12-17 | Data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112559808B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799647A (en) * | 2021-04-12 | 2021-05-14 | 四川新网银行股份有限公司 | Business personnel-oriented rule engine adapter device identification method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927174A (en) * | 2014-04-17 | 2014-07-16 | 北京视博数字电视科技有限公司 | Service processing method and device |
| EP2860629A1 (en) * | 2013-10-10 | 2015-04-15 | Alcatel Lucent | Method, processing framework, aggregator and computer program product for processing data |
| CN108881328A (en) * | 2018-09-29 | 2018-11-23 | 北京东土军悦科技有限公司 | Packet filtering method, device, gateway and storage medium |
| CN110020773A (en) * | 2019-01-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of data processing method, equipment and computer-readable medium |
| US20190342178A1 (en) * | 2018-05-07 | 2019-11-07 | Vmware, Inc. | Distributed rules engine for processing events associated with internet of things devices |
| CN110851283A (en) * | 2019-11-14 | 2020-02-28 | 百度在线网络技术（北京）有限公司 | Resource processing method and device and electronic equipment |
| CN110995846A (en) * | 2019-12-10 | 2020-04-10 | 北京数知科技股份有限公司 | Internet of things collected data processing device, method and system |
| CN111882134A (en) * | 2020-08-04 | 2020-11-03 | 山东大学 | Cloud computing service scheduling method, system, medium and electronic device |
-
2020
- 2020-12-17 CN CN202011497745.5A patent/CN112559808B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2860629A1 (en) * | 2013-10-10 | 2015-04-15 | Alcatel Lucent | Method, processing framework, aggregator and computer program product for processing data |
| CN103927174A (en) * | 2014-04-17 | 2014-07-16 | 北京视博数字电视科技有限公司 | Service processing method and device |
| US20190342178A1 (en) * | 2018-05-07 | 2019-11-07 | Vmware, Inc. | Distributed rules engine for processing events associated with internet of things devices |
| CN108881328A (en) * | 2018-09-29 | 2018-11-23 | 北京东土军悦科技有限公司 | Packet filtering method, device, gateway and storage medium |
| CN110020773A (en) * | 2019-01-30 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of data processing method, equipment and computer-readable medium |
| CN110851283A (en) * | 2019-11-14 | 2020-02-28 | 百度在线网络技术（北京）有限公司 | Resource processing method and device and electronic equipment |
| CN110995846A (en) * | 2019-12-10 | 2020-04-10 | 北京数知科技股份有限公司 | Internet of things collected data processing device, method and system |
| CN111882134A (en) * | 2020-08-04 | 2020-11-03 | 山东大学 | Cloud computing service scheduling method, system, medium and electronic device |
Non-Patent Citations (1)
| Title |
|---|
| 任宪臻;朱义;: "基于规则引擎的大规模网页信息抽取平台设计与实现", 北京城市学院学报, no. 05, pages 73 - 76 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799647A (en) * | 2021-04-12 | 2021-05-14 | 四川新网银行股份有限公司 | Business personnel-oriented rule engine adapter device identification method |
| CN112799647B (en) * | 2021-04-12 | 2021-11-23 | 四川新网银行股份有限公司 | Business personnel-oriented rule engine adapter device identification method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112559808B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210397947A1 (en) | Method and apparatus for generating model for representing heterogeneous graph node | |
| CN111753997B (en) | Distributed training method, system, device and storage medium | |
| WO2022000802A1 (en) | Deep learning model adaptation method and apparatus and electronic device | |
| JP7386827B2 (en) | Semantic processing methods, devices, electronic equipment and media | |
| CN110727434B (en) | Rendering method, rendering device, electronic equipment and storage medium | |
| CN111639027B (en) | Test method and device and electronic equipment | |
| CN112069201A (en) | Target data acquisition method and device | |
| JP2021197159A (en) | Method for pre-training graph neural network, program, and device | |
| CN112000450A (en) | Neural network architecture searching method and device | |
| CN111241234B (en) | Text classification method and device | |
| CN110619002A (en) | Data processing method, device and storage medium | |
| CN111460116B (en) | Question answering method, question answering system, electronic device and storage medium | |
| CN112491617B (en) | Link tracking method, device, electronic equipment and medium | |
| CN110852449A (en) | Model migration method and electronic device | |
| CN112527474A (en) | Task processing method and device, equipment, readable medium and computer program product | |
| CN111178937B (en) | User rewarding method and device for application, electronic equipment and readable storage medium | |
| CN112532720A (en) | Data forwarding method, device, equipment and storage medium | |
| CN110545324B (en) | Data processing method, device, system, network equipment and storage medium | |
| CN113961318A (en) | Distributed scheduling method, device, equipment and storage medium | |
| CN110659330A (en) | Data processing method, device and storage medium | |
| CN112559808B (en) | Data processing method and device and electronic equipment | |
| CN112422412B (en) | Information processing method, apparatus, device and medium | |
| CN111680599B (en) | Face recognition model processing method, device, equipment and storage medium | |
| WO2023169193A1 (en) | Method and device for generating smart contract | |
| CN111767149A (en) | Scheduling method, device, equipment and storage equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |