CN113868299A - Data processing method executed by rule engine and device of rule engine - Google Patents

Data processing method executed by rule engine and device of rule engine Download PDF

Info

Publication number
CN113868299A
CN113868299A CN202111143873.4A CN202111143873A CN113868299A CN 113868299 A CN113868299 A CN 113868299A CN 202111143873 A CN202111143873 A CN 202111143873A CN 113868299 A CN113868299 A CN 113868299A
Authority
CN
China
Prior art keywords
data
data processing
device access
access unit
processing method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111143873.4A
Other languages
Chinese (zh)
Inventor
张扬扬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111143873.4A priority Critical patent/CN113868299A/en
Publication of CN113868299A publication Critical patent/CN113868299A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24564Applying rules; Deductive queries

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computer And Data Communications (AREA)

Abstract

The embodiment of the disclosure discloses a data processing method executed by a rule engine, a device of the rule engine, an electronic device and a storage medium. The disclosure relates to the field of Internet of things and cloud computing, in particular to a rule engine technology of an Internet of things platform. The method of the embodiment of the present disclosure includes: receiving first data in a first data format via a first device access unit; receiving second data in a second data format via a second device access unit; performing data conversion on the second data to obtain converted second data with the first data format; forwarding the converted second data to the first device access interface; and forwarding, by the first device access unit, the first data and the converted second data to a data processing unit, respectively. By utilizing the embodiment provided by the disclosure, the data source supported by the rule engine can be conveniently expanded on the basis of the existing rule engine.

Description

Data processing method executed by rule engine and device of rule engine
Technical Field
The embodiment of the disclosure relates to the field of internet of things and cloud computing, in particular to a rule engine technology of an internet of things platform, and specifically relates to a data processing method executed by a rule engine, a rule engine device, electronic equipment and a storage medium.
Background
Under the scene of the rule engine service of the internet of things, most of current internet of things platforms are limited to use a device theme (topic) corresponding to the device access service set by the platform as an indication of a data source. The data sources that the platform of the internet of things can support are associated with the device access service provided by the platform of the internet of things.
Disclosure of Invention
According to an aspect of exemplary embodiments of the present disclosure, there is provided a data processing method performed by a rules engine, including: receiving first data in a first data format via a first device access unit; receiving second data in a second data format via a second device access unit; performing data conversion on the second data to obtain converted second data with the first data format; forwarding the converted second data to the first device access unit; and forwarding, by the first device access unit, the first data and the converted second data to a data processing unit, respectively.
According to another aspect of exemplary embodiments of the present disclosure, there is provided an apparatus of a rules engine, including: a first device access unit configured to receive first data in a first data format; a second device access unit configured to receive second data in a second data format; a format conversion unit configured to perform data conversion on the second data to obtain converted second data having the first data format; a forwarding unit configured to forward the converted second data to the first device access unit; and a data processing unit configured to receive the first data and the converted second data from the first device access unit, respectively.
According to another aspect of an exemplary embodiment of the present disclosure, an electronic device is provided. The electronic device includes: a processor and a memory storing a program. The program comprises instructions which, when executed by a processor, cause the processor to perform a method according to some exemplary embodiments of the present disclosure.
According to another aspect of exemplary embodiments of the present disclosure, there is provided a computer-readable storage medium storing a program. The program comprises instructions which, when executed by a processor of the electronic device, cause the processor to perform a method according to some exemplary embodiments of the present disclosure.
According to another aspect of exemplary embodiments of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements a method according to some exemplary embodiments of the present disclosure.
By means of the data processing method executed by the rule engine and the rule engine device, a data source access unit of the rule engine can be expanded on the basis of an existing internet of things rule engine, so that data sources which can be supported by the internet of things rule engine can be increased under the condition that the existing design is not changed, and the service quality of the rule engine is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements:
FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an embodiment of the present disclosure;
FIG. 2 illustrates an exemplary flow diagram of a data processing method performed by a rules engine according to an embodiment of the present disclosure;
FIG. 3 shows a schematic diagram of an application scenario according to an embodiment of the present disclosure;
FIG. 4 illustrates an exemplary block diagram of an apparatus of a rules engine in accordance with an embodiment of the present disclosure; and
fig. 5 shows a schematic block diagram of an example computing device, according to an example embodiment of the present disclosure.
Detailed Description
In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, the timing relationship, or the importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.
The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
In the scene of the internet of things, a user uses a rule engine service to process and forward the data of the internet of things equipment. Currently, most internet of things platforms are limited to use a device theme (topic) corresponding to a device access service of the internet of things platform as a data source, wherein the device theme is used for communication of internet of things devices at a device side and a server side. Therefore, the data source is strongly bound with the device access service provided by the platform of the internet of things, and the rule engine is difficult to flexibly support data access of different data sources. In terms of data types, most internet of things platforms only support processing of a single type of data (such as a Json type). However, the internet of things device data may have multiple types, such as Json type, binary type, and the like. How to support various types of data is also a problem to be solved by the internet of things rule engine service. The patent provides a universal solution, supports multiple data sources, and can meet the requirements of users on multiple data types.
In order to solve the above problems, the present disclosure provides a new data processing method for a rule engine, which enables to extend data sources or data types supported by the rule engine.
Embodiments of the present disclosure are further described below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or more internet of things devices 101, an internet of things platform 120, and one or more communication networks 110 coupling the one or more internet of things devices to the internet of things platform 120. The internet of things device 101 may send data generated or associated with the internet of things device during operation to the internet of things platform for further processing.
A rules engine according to embodiments of the present disclosure may be located at the internet of things platform 120. When the internet of things device 101 sends data to the internet of things platform 120, the rule engine performs data preprocessing on the data based on a predefined rule, data processing or data forwarding based on a mode specified by a user, and the like.
In some embodiments, the internet of things platform 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, for example, provided to the internet of things devices 101 under a software as a service (SaaS) model.
In the configuration shown in fig. 1, the internet of things platform 120 may include one or more components that implement the functions performed by the internet of things platform 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating the internet of things device 101 may, in turn, utilize one or more applications to interact with the internet of things platform 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. By way of example only, one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.
The internet of things platform 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The internet of things platform 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for servers).
The computing units in the internet of things platform 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.
In some implementations, the internet of things platform 120 can include one or more applications to analyze and consolidate data feeds and/or event updates received from the internet of things devices 101. The internet of things platform 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of the internet of things device 101.
In some embodiments, the internet of things platform 120 may be a server of a distributed system, or a server incorporating a blockchain. The internet of things platform 120 may also be a cloud server, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store data associated with devices collected by internet of things devices. The database 130 may reside in various locations. For example, the database used by the internet of things platform 120 may be local to the internet of things platform 120, or may be remote from the internet of things platform 120 and may communicate with the internet of things platform 120 via a network-based or dedicated connection. The database 130 may be of different types. In some embodiments, the database used by the internet of things platform 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to the command.
In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or regular stores supported by a file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
FIG. 2 illustrates an exemplary flow diagram of a data processing method performed by a rules engine according to an embodiment of the present disclosure. The method 200 described in fig. 2 may be performed using the rules engine of the internet of things platform 120 shown in fig. 1.
As shown in fig. 2, in step S202, first data in a first data format may be received via a first device access unit. In step S204, second data in a second data format may be received via the second device access unit. In step S206, data conversion may be performed on the second data to obtain converted second data having the first data format. In step S208, the converted second data may be forwarded to the first device access unit. In step S210, the first data and the converted second data may be forwarded to the data processing unit by the first device access unit, respectively.
By using the method 200 provided in fig. 2, the types of data sources that can be supported by the rule engine can be extended without modifying the data access service of the rule engine of the original internet of things platform, so that the rule engine can more flexibly implement access to multiple data sources.
The principle of the embodiments of the present disclosure will be described in detail below.
In step S202, first data in a first data format may be received via a first device access unit.
The first device may be an internet of things terminal device, such as the internet of things device 101 shown in fig. 1. The first device access unit may be configured to receive data generated by the first device as a first data source from the internet of things device based on the user-configured rule, and execute corresponding processing logic for the data. A device theme may be used to represent a first device as a first data source. The first data format may include a first data source indicative of a first device and a first data processing rule associated with the first data source. Furthermore, the first data format may also include any required other items related to the first data processing rule.
In some embodiments, the first device may send, according to the configuration of the internet of things platform, data in the first data format that meets the processing needs of the rule engine to the first device access unit, and then the first device access unit may forward the received first data to the data processing unit of the rule engine for further processing. Because the first data from the first data source has a data format specified by the rules engine, the rules engine is able to directly process the first data from the first data source.
In step S204, second data in a second data format may be received via the second device access unit.
First data generated in the working process of the first device serving as the terminal device of the internet of things can be sent to other storage devices serving as second data sources. For example, the working data generated at the first device may be sent to other databases or middleware to facilitate reading or use of such working data by users of any third party outside the internet of things system. In this case, in order for the rule engine to use the working data sent to (or stored in) the second data source, the first device needs to read data from the second data source and process the read data into the first data format, and then send the first data format to the rule engine of the internet of things platform for processing. This increases the workload of the first device.
In some embodiments, the second data is generated by the first device after sending the first data to the second data source. The second data has a second data format configured at the second data source, where the second data format may include information of the second data source that provides the second data. Where the second data source is middleware, the second data source may be represented by a middleware theme.
In some embodiments, the rules engine may access or establish a data connection with the second data source based on the user-provided configuration information to receive the second data from the second data source. For example, the rules engine may access or establish a data connection with the second data source based on configuration information such as an internet protocol address, username, password, etc. provided by the user.
In some implementations, a connectivity check between the rules engine and the second data source may be performed prior to step S204.
To ensure that data can be successfully retrieved from the second data source based on the configuration information provided by the user, a connectivity check between the rules engine and the second data source may be performed after the user has configured the access information of the second data source. In the event that the connectivity check passes, it may be determined that the rules engine was able to successfully obtain data from the second data source. If the connectivity check fails, it means that the rules engine may not be able to successfully obtain data from the second data source and does not allow the user to save data at the second data source. In this case, a prompt for re-inputting configuration information for connecting to the second data source may be provided to the user, so as to expose the problem as early as possible, and avoid operation errors caused by failure to successfully obtain correct data from the second data source in the subsequent working process.
In some embodiments, the rules engine may subscribe to messages from the second data source. After establishing a data connection between the rule engine and the second data source based on the configuration information provided by the user, the second data source may push data that conforms to the push rule to a second device access unit of the rule engine.
In step S206, data conversion may be performed on the second data to obtain converted second data having the first data format.
Because the second data source may be a third party storage device (e.g., data middleware), the second data format of the second data at the second data source is typically different from the first data format of the first data. It is difficult for the rules engine to directly process the second data obtained from the second data source. In some examples, the second data source may be a data middleware such as a rockmq, Kafka, or RabbitMQ. With such data middleware, data generated at the first device can be conveniently sent to a rules engine of the internet of things platform and any other destination capable of processing the data.
In some embodiments, data transforming the second data may include associating the second data source with a second data processing rule. With the above method, the rule engine can determine the data processing manner for the second data source in the converted second data so that further processing can be achieved.
The data transformation of the second data may further comprise retrieving any required further items related to the second data processing rule from the second data. In addition, the data conversion can also include adding any necessary information required by the rule engine to the acquired second data, so that the rule engine can conveniently perform subsequent processing on the converted second data. In some implementations, the second data processing rule may be the same as the first data processing rule, and the converted second data has the same first data format as the first data.
In some embodiments, determining whether to retrieve data from the second data source may be responsive to whether the second data processing rule is enabled. In response to the second data processing rule being disabled, receiving data from the second data source via the second device access unit may be ceased. In some implementations, the message subscription service from the second data source can be triggered when the second data processing rule is enabled. Accordingly, when the second data processing rule is disabled, the message subscription service from the second data source may be cancelled. With such an approach, flexibility in enabling or disabling data reception from other data sources may be achieved, thereby enabling the rules engine to flexibly configure whether data is to be obtained from other data sources.
In step S208, the converted second data may be forwarded to the first device access unit.
As previously mentioned, the converted second data may have the same first data format as the first data or any other data format that the rules engine is capable of handling, and thus, by forwarding the converted second data to the first device access unit, the rules engine may multiplex the existing data access service while enabling access to data from the first data source and from the second data source as the other data source. According to an embodiment of the present disclosure, the second data source may be a storage device external to any internet of things system, and the data format of the second data source need not be configured to the format required by the rules engine.
In step S210, the first data and the converted second data may be forwarded to the data processing unit by the first device access unit, respectively.
In some embodiments, the first data may have a first data type and the second data may have a second data type different from the first data type. For example, the first data type may be a Json data type and the second data type may be a binary data type. The first data type and the second data type may be any data type without departing from the principles of the present disclosure.
In some embodiments, the data processing unit may perform processing of the first data based on a first data processing rule associated with the first data and using a data processing statement of the first data type. Similarly, the data processing unit may enable processing of the converted second data based on a second data processing rule associated with the second data and with a data processing statement of the second data type. In some implementations, processing the first data and the converted second data can include determining statistics of the first data, such as a mean, median, sum, and so forth. In other implementations, processing the first data and the converted second data may include pass-through processing the first data and the converted second data. Pass-through processing refers to forwarding the raw data received by the rule engine directly to the next destination node. The data processing of the data is effected by the next destination node. In some examples, the type of data to be transparently transferred may be designated as the transparent transfer type, and the data processing statements for the data of the transparent transfer type may be designated as processing statements for forwarding to the next destination node.
In some implementations, a user can write and debug data processing statements for a first data type and data processing statements for a second data type using an interactive interface provided by the internet of things platform. When the rule engine needs to process the currently accessed data, the corresponding data processing statement can be called according to the data type of the currently processed data to realize data processing.
In some embodiments, after processing the first data and the converted second data with the data processing unit, the method 200 may further include sending the processed first data to a first destination node and sending the processed second data to a second destination node. In some implementations, the first destination node and/or the second destination node can be at least one of another data processing unit of the internet of things platform, a data storage device of the physical network platform, and an external data storage device.
By using the data processing method executed by the rule engine provided by the embodiment of the disclosure, data access and data processing which enable the rule engine to support multiple data sources and/or multiple data types can be realized. The message middleware (such as Kafka middleware) can be used as an intermediate layer between the terminal equipment of the internet of things and the rule engine of the internet of things, and provides a method for accessing data of the terminal equipment of the internet of things to the rule engine of the internet of things through the message middleware. Message middleware such as Kafka has the benefits of high throughput, high reliability, and the possibility of repeated consumption of data. In addition, the data reading mode of other data sources can be unified by the method of forwarding the data of all possible other data sources which are not the device subjects to the message middleware. In addition, by establishing a conversion relationship between the data format provided by the message middleware and the data format processed by the rule engine, the rule engine is enabled to extend the support for more types of data sources over existing designs. Furthermore, by providing data processing statements for different data types, the range of data types that can be processed by the rules engine can be extended so that the rules engine can process data of different data types.
Fig. 3 shows a schematic diagram of an application scenario according to an embodiment of the present disclosure.
As shown in FIG. 3, the rules engine 300 may include a control plane 310 and a data plane 320.
Control plane 310 may include a data source 311, data processing logic 312, a data destination 313, and a data processing statement debugging unit 314.
The control plane 310 may be used for user configuration rules. The user-configured rules may include data source, data type, data processing logic statements, data forwarding destination, and the like.
Data source 311 in control plane 310 supports user configuration of a variety of data sources. In some examples, a user may configure data sources outside of the device theme. For example, the user may edit information about the saved data sources (e.g., information about establishing a data connection with the Kafka middleware) to enable the rules engine to obtain more data from the data sources. Specifying the middleware theme that the rule needs to consume. In addition, connectivity checks are also supported for data sources compiled in data source 311.
Data processing logic 312 may include data processing statements for processing data of different data types, such as data processing statements for processing data of the Json data type, binary data type, or pass-through type. The user may select the data types that the rules engine is capable of supporting and write corresponding data processing statements for the supported data types. For example, a user may write data processing statements for Json-type data, data processing statements for binary data types, and data processing statements for pass-through type data for direct forwarding.
The data processing statement debugging unit 314 may be configured to debug the data processing statement edited by the user, so that the data processing statement can complete a desired data processing operation. The user may debug the data processing statements at a debug interface. By acquiring the original data, the data type and the processing statement input by the user in the debugging interface, the data processing statement debugging unit can display the processed data in the result bar. If the processing statement written by the user has a problem, the data processing statement debugging unit can also perform exception prompting, so that the user can conveniently check the effectiveness and effect of the data processing statement.
Data destination 313 can support editing information that holds a destination that will further send processed data after data processing is completed. The data destination may include a data processing unit of the internet of things platform, a data storage device of the internet of things platform, or an external storage device.
The data plane can be used for carrying out data processing on the data of the Internet of things equipment. According to the rules configured by the user in the control plane, the rule engine can receive the data corresponding to the data source in the data plane, execute corresponding data processing operation according to the data type, and forward the processed data to the corresponding data destination.
In the data plane 320, device X324 may send data to a first device access unit 326 of the rules engine via a message center 325. Wherein the message center 325 may be a Broker message center. Message center 325 may process the data provided by device X into a first data format that the rules engine can read and process. For example, the first data format may include a first data source corresponding to device X and a first data processing rule associated with the first data source.
When the first device access unit is used for acquiring the corresponding device theme as the data source, the subscription function for the device theme can be triggered or cancelled. When the user has configured the data source of the device topic and enables the data processing rule for the data source of the device topic, the message subscription service for the device access service of the device topic can be triggered. The device topic is associated with the data processing rule through the device access service, and data of the data source of the device topic can be pushed to the data processing unit for corresponding processing. The message subscription service for the data source of the device topic may be cancelled when the data processing rules for the message topic are disabled.
Device a 321 may send data to middleware 322 (e.g., Kafka middleware). The rules engine 300 may receive data from the middleware 322 using the second device access unit 323. Since the data retrieved from the middleware 322 may have a second data format different from the first data format. The rules engine 300 may not be able to directly read the data in the second data format. The data received from the second device access unit may be format converted by the format conversion unit to convert the data in the second data format into the first data format, i.e. a data format that the rule engine is capable of handling, before forwarding the data received from the middleware 322 to the first device access unit. The converted data having the first data format may then be forwarded to the first device access unit.
In some examples, the second device access unit may be developed as a generic message subscription service module. The universal message subscription service module can subscribe data from the middleware and convert the data sent by the middleware into a data format which can be processed by the rule engine. In some examples, the generic message subscription service module may also send the required information to the middleware.
After the user configures the data source for the middleware topic and enables the corresponding data processing rule at the control plane, the general message subscription service may associate the data from the data source for the middleware topic with the corresponding data processing rule. When there is a message in the configured middleware theme, the second device access unit may acquire the message and process the message according to the associated data processing rule. Subscribing to messages from the data source of the middleware topic may be cancelled when the data processing rule corresponding to the data source of the middleware topic is disabled.
The universal message subscription service module can provide a data publishing and consuming function of a unified middleware (taking Kafka as an example), and both a user and a rule engine service can use the universal module, so that the problem caused by inconsistent operations of the user and the rule engine service is prevented, and the subsequent unified adjustment of related operations is facilitated. Further, the module may obtain a mapping of data processing rules corresponding to the Kafka data. Therefore, the format conversion unit can associate the relevant Kafka messages with the data processing rules, and data can be conveniently processed according to the data types and the data processing logic subsequently.
Through further development of the original rule engine client on the data plane, the second device access unit can be utilized to forward data received by the rule engine from other data sources (such as a data source of a middleware theme) except the device theme to the device access unit of the device theme, so that the data sources which can be supported by the rule engine can be expanded without modifying the original device access unit. The second device access unit and the format conversion unit may be developed as a separately deployed service in the rules engine client. By format converting the data of the second data source, the original device access unit (i.e. the first device access unit) in the rule engine client can be multiplexed without modifying the first device access unit.
The data processing unit 328 may perform a data processing operation corresponding to a data processing rule on the data received by the rule engine to obtain a data processing result desired by the user. Data processing engine 327 may include data processing statements for different data types. Based on the data type of the data from the device a or the device X, a data processing statement of the corresponding data type in the data processing engine may be called to implement the corresponding data processing operation.
The processed user-desired data processing results may be sent to destination 329. The data processing process of the rule engine is ended up to this point.
According to the method for processing the rule engine service of the internet of things, the user is allowed to process data of other data sources by using the rule engine service of the internet of things through configuration, and the requirement that the rule engine service supports various data sources is well met. And allowing a user to select a data type according to the service requirement, writing a corresponding data processing statement, supporting debugging and storing after debugging. Meanwhile, the method executed by the rule engine according to the embodiment of the disclosure has good expandability, such as no modification to the existing data source receiving module is required, and other types of data processing are allowed to be subsequently expanded.
FIG. 4 illustrates an exemplary block diagram of an apparatus of a rules engine in accordance with an embodiment of the disclosure.
As shown in fig. 4, the apparatus 400 of the rule engine may include a first device access unit 410, a second device access unit 420, a format conversion unit 430, a forwarding unit 440, and a data processing unit 450.
The first device access unit 410 may be configured to receive first data in a first data format. The second device access unit 420 may be configured to receive second data in a second data format. The format conversion unit 430 may be configured to perform data conversion on the second data to obtain converted second data having the first data format. The forwarding unit 440 may be configured to forward the converted second data to the first device access unit. The data processing unit 450 may be configured to receive the first data and the converted second data from the first device access unit, respectively.
The operations of the units 410-450 of the rule engine apparatus 400 are similar to the operations of the steps S202-S210, and are not repeated herein.
According to an embodiment of the present disclosure, there is also provided an electronic apparatus including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in connection with fig. 2.
There is also provided, in accordance with an embodiment of the present disclosure, a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method described in connection with fig. 2.
There is also provided, in accordance with an embodiment of the present disclosure, a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, implements the method described in connection with fig. 2.
Referring to fig. 5, a block diagram of an electronic device 500 that may be a rules engine of the internet of things platform of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic apparatus 500 can also be stored. The calculation unit 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
A number of components in the electronic device 500 are connected to the I/O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information to the electronic device 500, and the input unit 506 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote controller. Output unit 507 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 508 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 509 allows the electronic device 500 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, such as bluetoothTMDevices, 1302.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like.
The computing unit 501 may be a variety of general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 501 performs the respective methods and processes described above, such as the data processing method according to the embodiment of the present disclosure. For example, in some embodiments, methods according to embodiments of the present disclosure may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured by any other suitable means (e.g., by means of firmware) to perform the methods of the embodiments of the present disclosure.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.

Claims (16)

1. A data processing method performed by a rules engine, comprising:
receiving first data in a first data format via a first device access unit;
receiving second data in a second data format via a second device access unit;
performing data conversion on the second data to obtain converted second data with the first data format;
forwarding the converted second data to the first device access unit; and
forwarding, by the first device access unit, the first data and the converted second data to a data processing unit, respectively.
2. The data processing method of claim 1, wherein the first data format comprises a first data source providing the first data and a first data processing rule associated with the first data source, and the second data format comprises a second data source providing the second data.
3. The data processing method of claim 2, wherein data converting the second data comprises:
associating the second data source with a second data processing rule.
4. The data processing method of claim 3, further comprising:
in response to the second data processing rule being disabled, ceasing to receive data from the second data source via the second device access unit.
5. The data processing method of claim 2, wherein the second data source is a data middleware.
6. The data processing method of claim 4, wherein the data middleware is a RockettMQ, Kafka, or RabbitMQ.
7. The data processing method of claim 2, wherein prior to receiving the second data in the second data format via the second device access unit, the method comprises:
performing connectivity checks between the rules engine and the second data source.
8. The data processing method of any of claims 1 to 7, wherein the first data is of a first data type and the second data is of a second data type different from the first data type.
9. The data processing method of claim 8, wherein the first data type is a Json data type and the second data type is a binary data type.
10. The data processing method of claim 8, further comprising:
processing, by the data processing unit, the first data with a data processing statement of a first data type based on a first data processing rule associated with the first data;
processing the converted second data with a data processing statement of a second data type is effected by the data processing unit based on a second data processing rule associated with the second data.
11. The data processing method of claim 10,
the data processing unit realizes processing of the first data by using a data processing statement of a first data type, and the data processing unit performs transparent transmission processing on the first data;
processing, by the data processing unit, the second data using the data processing statement of the second data type includes performing, by the data processing unit, pass-through processing on the converted second data.
12. The data processing method of claim 10, further comprising:
sending the processed first data to a first destination node;
the processed second data is sent to the second destination node.
13. An apparatus of a rules engine, comprising:
a first device access unit configured to receive first data in a first data format;
a second device access unit configured to receive second data in a second data format;
a format conversion unit configured to perform data conversion on the second data to obtain converted second data having the first data format;
a forwarding unit configured to forward the converted second data to the first device access unit; and
a data processing unit configured to receive the first data and the converted second data from the first device access unit, respectively.
14. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-12.
15. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the data processing method according to any one of claims 1 to 12.
16. A computer program product comprising a computer program, wherein the computer program realizes the data processing method of any one of claims 1-12 when executed by a processor.
CN202111143873.4A 2021-09-28 2021-09-28 Data processing method executed by rule engine and device of rule engine Pending CN113868299A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111143873.4A CN113868299A (en) 2021-09-28 2021-09-28 Data processing method executed by rule engine and device of rule engine

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111143873.4A CN113868299A (en) 2021-09-28 2021-09-28 Data processing method executed by rule engine and device of rule engine

Publications (1)

Publication Number Publication Date
CN113868299A true CN113868299A (en) 2021-12-31

Family

ID=78991863

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111143873.4A Pending CN113868299A (en) 2021-09-28 2021-09-28 Data processing method executed by rule engine and device of rule engine

Country Status (1)

Country Link
CN (1) CN113868299A (en)

Similar Documents

Publication Publication Date Title
CN112615746B (en) Edge calculation method, device and system
CN115292026B (en) Management method, device and equipment of container cluster and computer readable storage medium
CN108776587B (en) Data acquisition method and device, computer equipment and storage medium
CN113393367B (en) Image processing method, apparatus, device and medium
CN111367635A (en) Method, apparatus, device and medium for implementing application interface in host platform layer
CN113050940A (en) Method for previewing small program, related device and computer program product
CN114265680A (en) Mass data processing method and device, electronic equipment and storage medium
US9208058B2 (en) Providing directional debugging breakpoints
CN115329170A (en) Webpage crawling method, device, equipment and storage medium
CN113326403A (en) Flowchart rendering method and device, electronic equipment and medium
WO2023169193A1 (en) Method and device for generating smart contract
CN113868299A (en) Data processing method executed by rule engine and device of rule engine
CN114510308B (en) Method, device, equipment and medium for storing application page by mobile terminal
CN114070889B (en) Configuration method, traffic forwarding device, storage medium, and program product
CN113986401A (en) Data processing method and device, electronic equipment and storage medium
CN112558968B (en) Method, device, equipment and storage medium for generating resource tree view
CN113079046A (en) Data access method and device, electronic equipment and medium
CN114095564A (en) Data processing method and device, equipment and medium
CN112988482A (en) Server operation management method, device, terminal, server and storage medium
JP2022525568A (en) How to implement application interfaces on the upper platform layer, equipment, equipment and media
CN110768855A (en) Method and device for testing linkmzation performance
CN115454647A (en) Data processing method and device, equipment and medium
CN113010383B (en) Error log filtering method, device, electronic equipment and readable medium
CN116225576B (en) Application program data environment switching method and device, electronic equipment and medium
CN115277652B (en) Streaming media processing method and device based on reasoning service and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination