CN116303418A - Mass rule processing method, system and storage medium based on Internet of things platform - Google Patents
Mass rule processing method, system and storage medium based on Internet of things platform Download PDFInfo
- Publication number
- CN116303418A CN116303418A CN202211644686.9A CN202211644686A CN116303418A CN 116303418 A CN116303418 A CN 116303418A CN 202211644686 A CN202211644686 A CN 202211644686A CN 116303418 A CN116303418 A CN 116303418A
- Authority
- CN
- China
- Prior art keywords
- rule
- service
- hash
- configuration information
- distribution management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a mass rule processing method, a mass rule processing system and a mass rule processing storage medium based on an Internet of things platform. The method belongs to the technical field of the Internet of things, and comprises the following steps: the rule distribution management service calculates rule hash values of all rules according to the acquired user identifications and stores the rule hash values into a database rule table; the rule distribution management service generates a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to a preset hash partition rule, stores the rule configuration information to obtain, sends the rule configuration information to the corresponding rule processing service, and sends the total rule configuration information to the message dispatching service; the rule processing service obtains rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rules; and the message dispatch service distributes the received reported information according to the full rule configuration information and the user identification in the reported information. The embodiment of the application can improve the processing efficiency of the mass rules.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to a mass rule processing method, a mass rule processing system and a mass rule storage medium based on an Internet of things platform.
Background
The internet of things platform generally has general basic capabilities such as terminal access, terminal management, message pushing, rule engine and the like. After the internet of things terminal is accessed to the internet of things platform, a message pushing module pushes the message reported by the terminal to the internet of things application according to a user message pushing rule; or the rule engine module cleans, filters and processes the information reported by the terminal based on the user customized processing rule.
The message pushing module and the rule engine module of the internet of things platform generally load rules configured by a user into a memory to improve the rule calculation processing efficiency. Along with the large-scale development of the internet of things, the number of terminals and applications accessing to the internet of things platform is continuously increased, and the number of messages reported by the terminals, message pushing rules customized by users and rule engine processing rules are continuously increased. The rule processing service is deployed in a cluster mode, and generally, the reported message is randomly sent to a rule processing service for processing, so that in order to ensure normal service, the rule processing service needs to load the whole amount of rules into the memory, and the loading of the mass rule memory can cause the memory capacity of the server to be limited, so that the rule processing efficiency is reduced.
Disclosure of Invention
The embodiment of the invention provides a mass rule processing method, a system and a storage medium based on an Internet of things platform, aiming at improving the processing efficiency of mass rules in the existing Internet of things platform.
In a first aspect, an embodiment of the present invention provides a method for processing a mass rule based on an internet of things platform, including:
the rule distribution management service obtains a user identifier from a database rule table, calculates a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and stores the rule hash value into the database rule table;
the rule distribution management service generates a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information;
the rule distribution management service sends the rule configuration information to the corresponding rule processing service and sends the total rule configuration information to a message dispatching service;
the rule processing service obtains rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rules;
And if the message dispatch service receives the report information, distributing the report information according to the full rule configuration information and the user identification in the report information.
In a second aspect, an embodiment of the present invention further provides a mass rule processing system based on an internet of things platform, which includes: an acquisition computing unit and a generation first transmitting unit configured in a rule distribution management service, a first acquisition loading unit configured in a rule processing service and an allocation unit configured in a message dispatch service, wherein,
the obtaining and calculating unit is used for obtaining a user identifier from a database rule table by the rule distribution management service, calculating a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and storing the rule hash value into the database rule table;
the generation and storage unit is used for the rule distribution management service to generate a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information;
the first sending unit is used for sending the rule configuration information to the corresponding rule processing service by the rule distribution management service and sending the total rule configuration information to a message dispatch service;
The first obtaining and loading unit is used for obtaining rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information by the rule processing service for loading;
and the distribution unit is used for distributing the reported information according to the full rule configuration information and the user identification in the reported information if the reported information is received by the message dispatch service.
In a third aspect, an embodiment of the present invention further provides a massive rule processing system based on an internet of things platform, where the massive rule processing system includes a rule distribution management service, a rule processing service, and a message dispatching service, where each of the rule distribution management service, the rule processing service, and the message dispatching service includes a memory and a processor, where a computer program is stored in the memory, and the processor of the rule distribution management service, the rule processing service, and the message dispatching service implements the above method when executing the computer program.
In a fourth aspect, embodiments of the present invention also provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the above method.
The embodiment of the invention provides a mass rule processing method, a system and a storage medium based on an Internet of things platform. Wherein the method comprises the following steps: the rule distribution management service obtains a user identifier from a database rule table, calculates a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and stores the rule hash value into the database rule table; the rule distribution management service generates a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information; the rule distribution management service sends the rule configuration information to the corresponding rule processing service and sends the total rule configuration information to a message dispatching service; the rule processing service obtains rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rules; and if the message dispatch service receives the report information, distributing the report information according to the full rule configuration information and the user identification in the report information. According to the technical scheme of the embodiment of the invention, the rule distribution management service generates a rule hash partition list to be loaded by each rule processing service according to the preset rule Ha Xishang limit value, and stores the rule hash partition list to obtain rule configuration information, so that each rule processing service only needs to load a part of rules into a memory without loading the whole rule; when the message dispatch service receives the report information, the report information is distributed according to the full rule configuration information and the user identification in the report information, so that only the rule processing service with the user identification processes the report information, and the processing efficiency of massive rules can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a timing chart of a mass rule processing method based on an internet of things platform according to an embodiment of the present invention;
fig. 2 is a flow chart of a mass rule processing method based on an internet of things platform according to an embodiment of the present invention;
fig. 3 is a flow chart of a mass rule processing method based on an internet of things platform according to another embodiment of the present invention;
FIG. 4 is a schematic block diagram of a mass rule processing system based on an Internet of things platform according to an embodiment of the present invention; and
fig. 5 is a schematic block diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Referring to fig. 1, fig. 1 is a timing chart of a mass rule processing method based on an internet of things platform according to an embodiment of the present invention. The mass rule processing system based on the internet of things platform comprises a rule distribution management service 10, a rule processing service 20 and a message dispatch service 30. Wherein the rule distribution management service 10 communicates with the rule processing service 30 to issue rule configuration information to the rule processing service 20; the rule distribution management service 10 communicates with the message dispatch service 30 to send full-scale rule configuration information to the message dispatch service 30; the rule processing service 20 and the message dispatch service 30 receive the report information issued by the message dispatch service 30.
Fig. 2 is a flow chart of a mass rule processing method based on an internet of things platform provided by the embodiment of the invention. The mass rule processing method based on the Internet of things platform can be applied to a rule processing system, for example, the mass rule processing method based on the Internet of things platform can be realized through a software program configured on the rule processing system, so that the rule processing efficiency is improved. As shown in fig. 2, the method includes the following steps S110 to S150.
S110, the rule distribution management service obtains a user identifier from a database rule table, calculates a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and stores the rule hash value into the database rule table.
In the embodiment of the invention, a database rule table is created in a database service and mainly comprises fields such as a serial number, a user identifier, rule information, update time, hashTag and the like, wherein hashTag is used for storing hashtvalue corresponding to a rule, and the hashtvalue is calculated based on the user identifier. Specifically, the rule distribution management service obtains a user identifier from a database rule table, and calculates a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value. More specifically, a user identifier is obtained, and a hash algorithm is used for carrying out hash calculation on the user identifier to obtain a first hash value; and performing residual calculation on the preset rule Ha Xishang limit value by using the first hash value with the absolute value to obtain a second hash value, and taking the second hash value as a rule hash value. For ease of understanding, assume that the maximum number of rule processing service instances is SNmax; the preset rule Ha Xishang limit value is hashMax, in order to improve the subsequent rule distribution balance rate, rule processing service with a preset rule Ha Xishang limit value not smaller than 10 times is set, that is, hashMax > =10×snmax is set, the user identifier is uId, the rule hash value of each rule is hashvue= |hashcode (uId) |hashmax, where hashvue value takes a value space of [0, hashMax), hashCode (uId) is the first hash value, and|hashcode (uId) | is the first hash value after taking an absolute value. After the rule hash value is calculated, the rule hash value is stored in a hashTag field in the database rule table. It should be noted that, in this embodiment, the rule corresponding to the hashvue value is stored in the rule information field in the database rule table, and it is to be understood that in this embodiment, the hashvue value needs to be calculated, so as to facilitate the subsequent segmentation and partitioning of the hashvue value.
S120, the rule distribution management service generates a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information.
In the embodiment of the invention, the rule distribution management service identifies n rule processing services and divides the range of the preset rule Ha Xishang limit value into M rule hash partition lists, wherein M is more than or equal to n; configuring m rule hash partition lists to n rule processing services in a banding method; and storing the corresponding relation between the rule processing service and the rule hash partition list for each rule processing service to obtain rule configuration information. For easy understanding, the steps for generating the rule hash partition list will now be specifically described as follows:
(a) Identifying the rule processing services, and assuming that n rule processing services exist, identifying each rule processing service as S0, S1, S2.
(b) The hashvue value space [0, hashMax ] is segmented into M hashRange, where M > =snmax, hashMax is the preset rule Ha Xishang limit, hashRange is the rule hash partition list, and as can be understood,
hashRange[0]＝[0,hashMax/M]；
hashRange [1] = [ hashMax/m+1,2 x hashMax/M ]; and so on, up to hashRange [ M ].
(c) M hashRange is configured to n rule processing services in a striped manner, namely hashRange [0] is assigned to S0, hashRange [1] is assigned to S1, hashRange [ n ] is assigned to Sn, hashRange [ n+1] is circularly assigned to S1, and so on until hashRange [ M ] configuration is completed, each rule processing service is assigned to at least one hashRange list.
(d) And storing each rule processing service and the hashRange list relation thereof as rule configuration information.
In this embodiment, a rule processing service list is set in the rule distribution management service, and n rule processing service identifiers are managed through the rule processing service list, so that the capacity expansion of the subsequent rule processing service is facilitated; the rule is segmented and fragmented to obtain M hashRanges, and the hashRanges corresponding to each processing service are distributed, so that each rule processing service only needs to load a part of rules to a memory, and does not need to load the whole amount of rules, and the memory capacity limitation of the rule processing service when processing massive rules is avoided.
And S130, the rule distribution management service sends the rule configuration information to the corresponding rule processing service and sends the total rule configuration information to a message dispatching service.
In the embodiment of the invention, after the rule distribution management service calculates rule configuration information of each rule processing service, the rule configuration information is sent to the corresponding rule processing service, and the total rule configuration information is sent to the message dispatching service. Understandably, the full rule configuration information is a sum of the rule configuration information.
And S140, the rule processing service acquires the rule corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rule.
In the embodiment of the invention, the rule processing service obtains the rule corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rule. Specifically, the rule processing service obtains the rule hash partition list from the rule configuration information; and acquiring a rule corresponding to the rule hash partition list from a preset field of the database rule table for loading, wherein the preset field is hashTag. Understandably, the rule hash partition list includes a plurality of hashvue values, and the rule processing service can acquire a plurality of rules from the hashvue values to load.
And S150, if the message dispatch service receives the report information, distributing the report information according to the full rule configuration information and the user identification in the report information.
In the embodiment of the invention, if the message dispatch service receives the report information sent by the terminal of the internet of things, the message dispatch service acquires the user identifier from the report information, and carries out hash calculation on the user identifier through a hash algorithm to obtain the rule hash value; comparing the rule hash value with the rule hash partition list in the full rule configuration information one by one to determine the rule hash partition list in which the rule hash value is located; and distributing the reported information to the rule processing service corresponding to the rule hash partition list. Specifically, the message dispatch service assigns the reported information as follows:
(a) Acquiring a user identifier uId from the reported message;
(b)hashValue＝|hashCode(uId)|％hashMax；
(c) Comparing the hashRange list of each rule processing service one by one through rule configuration information in the rule service, and inquiring rule processing service Sa of which the hashRange contains the hashValue;
(d) The message dispatch service sends the message to the rule processing service Sa for processing.
Fig. 3 is a flow chart of a mass rule processing method based on an internet of things platform according to another embodiment of the present invention, as shown in fig. 3, in this embodiment, the method includes steps S110 to S190. That is, in this embodiment, the method further includes steps S160 to S190 after step S150 of the above embodiment.
S160, the rule distribution management service expands the capacity of the rule processing service to obtain a change rule configuration information list;
s170, the rule distribution management service sends the change rule configuration information in the change rule configuration information list to the corresponding rule processing service, and sends the change total rule configuration information to the message dispatching service;
s180, the rule processing service acquires a rule corresponding to the rule hash value from the database rule table according to the change rule configuration information and loads the rule;
and S190, the message dispatch service updates the full-volume rule configuration information according to the content in the changed full-volume rule configuration information.
In the embodiment of the invention, if a rule processing service is newly added, the rule distribution management service expands the rule processing service to obtain a changed rule configuration information list, specifically, the rule distribution management service identifies the newly added rule processing service and calculates the number of average rule hash partition lists which can be borne by the newly added rule processing service; sorting the rule processing services from large to small according to the number of rule hash partition lists in each rule processing service to obtain a rule processing service sorting list; the rule distribution management service migrates a rule hash partition list on the rule processing service sequencing list meeting a preset condition to the newly added rule processing service according to the average rule hash partition list number until the rule hash partition list on the newly added rule processing service is not smaller than the average rule hash partition list number, wherein the preset condition is that the number of the rule hash partition lists on the rule processing service is larger than the average rule hash partition list number; and the rule distribution management service stores the rule configuration information of the rule processing service migrated through the rule hash partition list to obtain a change rule configuration information list. The rule distribution management service sends the change rule configuration information in the change rule configuration information list to the corresponding rule processing service, and sends the change total rule configuration information to the message dispatching service; the rule processing service obtains rules corresponding to the rule hash values from the database rule table according to the change rule configuration information and loads the rules; and the message dispatch service updates the full rule configuration information according to the content in the changed full rule configuration information. For easy understanding, a rule processing service is added, and the steps of the capacity expansion method are specifically described as follows:
(a) The newly added rule processing service is marked as Sx, and x is the total number of the current rule processing service and is added with 1;
(b) Adding a newly added rule processing service Sx in a rule processing service list of the rule distribution management service;
(c) Calculating the average rule hash partition list number avgHashRange borne by the newly added rule processing services S1, S2 and the average rule hash partition list number avgHashRange borne by the Sx, wherein avgHashRange=M/total rule processing service number, and M is the number of hash Ranges;
(d) Sorting rule processing services from large to small according to hashRange numbers carried by each rule processing service to obtain a rule processing service sorting list listS;
(e) Starting from the first rule processing service in the rule processing service sequencing list listS, circularly migrating hashRange on the rule processing service sequencing list to Sx one by one until the condition fx is satisfied: hashRange number on Sx > = avgHashRange or hashRange number on the rule processing service < = avgHashRange;
(f) If the condition fx cannot be met, continuing to select the next rule processing service from the rule processing service ordered list listS, and performing the processing step in e) until fx is met or irregular processing service in listS;
(g) Marking the rule processing service with changed hashRange number to a list listSc, and processing configuration information by the changed rule;
(h) The rule processing configuration information of each rule processing service in listssc is issued to the corresponding rule processing service one by one, and all rule processing configuration information in listssc is issued to the message dispatching service;
(i) The rule processing service which receives the rule configuration information loads and updates the rule according to the received rule configuration information, and the message dispatching service updates the whole rule configuration information according to the rule content in listSc.
In this embodiment, the capacity expansion method can expand the capacity of the rule processing service, thereby further improving the rule processing efficiency. It should be noted that, in this embodiment, the database service, the rule distribution management service, the message dispatch service, and each rule processing service are all deployed on different virtual machines.
Fig. 4 is a schematic block diagram of a mass rule processing system 200 based on an internet of things platform according to an embodiment of the present invention. As shown in fig. 4, the mass rule processing system 200 based on the internet of things platform includes a unit for executing the mass rule processing method based on the internet of things platform described above, corresponding to the mass rule processing method based on the internet of things platform applied to the rule distribution management service 10, the rule processing service 20, and the message dispatch service 30 described above. Specifically, referring to fig. 4, the mass rule processing system 200 based on the internet of things platform includes an acquisition computing unit 101, a generation and storage unit 102 and a first sending unit 103 configured in the rule distribution management service 10, a first acquisition loading unit 201 configured in the rule processing service 20, and an allocation unit 301 configured in the message dispatch service 30.
The acquiring and calculating unit 101 is configured to acquire a user identifier from a database rule table by using a rule distribution management service, calculate a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and store the rule hash value in the database rule table; the generation and storage unit 102 is configured to generate, by using the rule distribution management service according to the preset hash partition rule, a rule hash partition list that needs to be loaded by each rule processing service in a striped manner, and store the rule hash partition list to obtain rule configuration information; the first sending unit 103 is configured to send the rule configuration information to the corresponding rule processing service by using the rule distribution management service, and send the full rule configuration information to a message dispatch service; the first obtaining and loading unit 201 is configured to obtain, by using the rule processing service according to the rule configuration information, a rule corresponding to the rule hash partition list from the database rule table, and load the rule; the allocation unit 301 is configured to allocate, if the message dispatch service receives the report information, the report information according to the full rule configuration information and the user identifier in the report information.
In some embodiments, for example, the acquiring computing unit 101 includes a first computing unit and a second computing unit.
The first computing unit is used for acquiring a user identifier, and carrying out hash computation on the user identifier through a hash algorithm to obtain a first hash value; the second calculation unit is configured to perform remainder calculation on a preset rule Ha Xishang limit value by using the first hash value after taking the absolute value to obtain a second hash value, and use the second hash value as a rule hash value.
In some embodiments, for example, the generating and saving unit 102 includes an identification unit, a configuration unit, and a first saving unit.
The identification unit is used for identifying n rule processing services by the rule distribution management service, and dividing the range of the preset rule Ha Xishang limit value into M rule hash partition lists, wherein M is more than or equal to n; the configuration unit is used for configuring m rule hash partition lists to n rule processing services in a banding method; the first storage unit is configured to store, for each rule processing service, a correspondence between the rule processing service and the rule hash partition list to obtain rule configuration information.
In some embodiments, for example, the first fetch load unit 201 includes a fetch unit and a load unit.
The obtaining unit is used for obtaining the rule hash partition list from the rule configuration information by the rule processing service; the loading unit is used for acquiring rules corresponding to the rule hash partition list from preset fields of the database rule table to load.
In some embodiments, for example, the allocation unit 301 includes a third calculation unit, an alignment unit, and an allocation subunit.
The third calculation unit is used for acquiring the user identifier from the reported information and carrying out hash calculation on the user identifier through a hash algorithm to obtain the rule hash value; the comparison unit is used for comparing the rule hash value with the rule hash partition list in the full rule configuration information one by one so as to determine the rule hash partition list in which the rule hash value is located; the allocation subunit is configured to allocate the reporting information to the rule processing service corresponding to the rule hash partition list.
The mass rule processing system 200 based on the platform of the internet of things according to another embodiment of the present invention is based on the above embodiment, and is added with the capacity expansion unit 104 and the second sending unit 105 configured in the rule distribution management service 10, the second obtaining and loading unit 202 configured in the rule processing service 20, and the updating unit 302 configured in the message dispatch service.
The capacity expansion unit 104 is configured to expand the capacity of the rule processing service by using the rule distribution management service to obtain a changed rule configuration information list; the second sending unit 105 is configured to send, by the rule distribution management service, change rule configuration information in the change rule configuration information list to the corresponding rule processing service, and send change total rule configuration information to the message dispatch service; the second obtaining and loading unit 202 is configured to obtain, by the rule processing service according to the change rule configuration information, a rule corresponding to the rule hash value from the database rule table and load the rule; the updating unit 302 is configured to update the full-dose rule configuration information according to the content in the modified full-dose rule configuration information by using the message dispatch service.
In some embodiments, for example, in this embodiment, the capacity expansion unit 104 includes an identifier calculation unit, a sorting unit, a migration unit, and a second storage unit.
The identification calculation unit is used for identifying the newly added rule processing service by the rule distribution management service and calculating the average rule hash partition list number which can be borne by the newly added rule processing service; the ordering unit is used for ordering the rule processing services from large to small according to the number of the rule hash partition lists in each rule processing service to obtain a rule processing service ordering list; the migration unit is configured to migrate, by the rule distribution management service, a rule hash partition list on the rule processing service sequencing list that meets a preset condition to the newly added rule processing service according to the average number of rule hash partition lists, until the rule hash partition list on the newly added rule processing service is not less than the average number of rule hash partition lists; the second storage unit is configured to store, by the rule distribution management service, rule configuration information corresponding to the rule processing service migrated through the rule hash partition list, to obtain a changed rule configuration information list.
It should be noted that, as those skilled in the art can clearly understand, the specific implementation process of the mass rule processing system 200 and the units based on the internet of things platform can refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, the description is omitted here.
The mass rule processing system based on the internet of things platform can be implemented in the form of a computer program, which can be run on a computer device as shown in fig. 5.
Referring to fig. 5, fig. 5 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 900 is a device in which a rule distribution management service, a rule processing service, and a message dispatch service are built.
With reference to fig. 5, the computer device 900 includes a processor 902, a memory and an interface 907 connected by a system bus 901, wherein the memory may include a storage medium 903 and an internal memory 904.
The storage medium 903 may store an operating system 9031 and a computer program 9032. The computer program 9032, when executed, may cause the processor 902 to perform a mass rule processing method based on an internet of things platform.
The processor 902 is operable to provide computing and control capabilities to support the operation of the overall computer device 900.
The internal memory 904 provides an environment for the execution of the computer program 9032 in the storage medium 903, which computer program 9032, when executed by the processor 902, may cause the processor 902 to execute a mass rule processing method based on the internet of things platform.
The interface 905 is used to communicate with other devices. Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of a portion of the architecture in connection with the present application and is not intended to limit the computer device 900 to which the present application is applied, and that a particular computer device 900 may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Wherein the processor 902 of each of the rule distribution management service, rule processing service and message dispatch service is configured to execute a computer program 9032 stored in a memory to implement the flow steps of the embodiments of the method described above.
It should be appreciated that in embodiments of the present application, the processor 902 may be a central processing unit (Central Processing Unit, CPU), the processor 902 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (terminals lication Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program may be stored in a storage medium that is a computer readable storage medium. The computer program is executed by at least one processor in the wireless communication system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program. The computer program, when executed by the processor, causes the processor to execute any embodiment of the mass rule processing method based on the internet of things platform.
The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, wireless communication software, or combinations of both, and that the various illustrative elements and steps have been described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the system of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal wireless communication, a terminal, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (10)
1. The mass rule processing method based on the platform of the Internet of things is characterized by comprising the following steps of:
the rule distribution management service obtains a user identifier from a database rule table, calculates a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and stores the rule hash value into the database rule table;
The rule distribution management service generates a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information;
the rule distribution management service sends the rule configuration information to the corresponding rule processing service and sends the total rule configuration information to a message dispatching service;
the rule processing service obtains rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information and loads the rules;
and if the message dispatch service receives the report information, distributing the report information according to the full rule configuration information and the user identification in the report information.
2. The mass rule processing method based on the internet of things platform according to claim 1, wherein the step of calculating a rule hash value according to the user identifier and a preset rule hash upper limit value comprises:
obtaining a user identifier, and carrying out hash calculation on the user identifier through a hash algorithm to obtain a first hash value;
and performing residual calculation on the preset rule Ha Xishang limit value by using the first hash value with the absolute value to obtain a second hash value, and taking the second hash value as a rule hash value.
3. The mass rule processing method based on the platform of the internet of things according to claim 1, wherein the rule distribution management service generates a rule hash partition list to be loaded by each rule processing service in a banding manner according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information, and the method comprises the steps of:
the rule distribution management service identifies n rule processing services and divides the range of the preset rule Ha Xishang limit value into M rule hash partition lists, wherein M is more than or equal to n;
configuring m rule hash partition lists to n rule processing services in a banding method;
and storing the corresponding relation between the rule processing service and the rule hash partition list for each rule processing service to obtain rule configuration information.
4. The mass rule processing method based on the platform of the internet of things according to claim 1, wherein the step of the rule processing service obtaining, from the database rule table according to the rule configuration information, a rule corresponding to the rule hash partition list for loading includes:
the rule processing service obtains the rule hash partition list from the rule configuration information;
And acquiring rules corresponding to the rule hash partition list from preset fields of the database rule table for loading.
5. The mass rule processing method based on the internet of things platform according to claim 1, wherein the step of distributing the report information according to the total rule configuration information and the user identifier in the report information comprises:
acquiring the user identifier from the reported information, and carrying out hash calculation on the user identifier through a hash algorithm to obtain the rule hash value;
comparing the rule hash value with the rule hash partition list in the full rule configuration information one by one to determine the rule hash partition list in which the rule hash value is located;
and distributing the reported information to the rule processing service corresponding to the rule hash partition list.
6. The mass rule processing method based on the internet of things platform according to claim 1, wherein the method further comprises:
the rule distribution management service expands the capacity of the rule processing service to obtain a change rule configuration information list;
the rule distribution management service sends the change rule configuration information in the change rule configuration information list to the corresponding rule processing service, and sends the change total rule configuration information to the message dispatching service;
The rule processing service obtains rules corresponding to the rule hash values from the database rule table according to the change rule configuration information and loads the rules;
and the message dispatch service updates the full rule configuration information according to the content in the changed full rule configuration information.
7. The mass rule processing method based on the platform of the internet of things according to claim 6, wherein the step of the rule distribution management service expanding the rule processing service to obtain the changed rule configuration information list includes:
the rule distribution management service identifies the newly added rule processing service and calculates the average rule hash partition list number which can be borne by the newly added rule processing service;
sorting the rule processing services from large to small according to the number of rule hash partition lists in each rule processing service to obtain a rule processing service sorting list;
the rule distribution management service migrates the rule hash partition list on the rule processing service sequencing list meeting the preset condition to the newly added rule processing service according to the average rule hash partition list number until the rule hash partition list on the newly added rule processing service is not smaller than the average rule hash partition list number;
And the rule distribution management service stores the rule configuration information of the rule processing service migrated through the rule hash partition list to obtain a change rule configuration information list.
8. The mass rule processing system based on the platform of the Internet of things is characterized by comprising: an acquisition calculation unit, a generation and storage unit and a first transmission unit which are configured in a rule distribution management service, a first acquisition loading unit configured in a rule processing service and an allocation unit configured in a message dispatch service, wherein,
the obtaining and calculating unit is used for obtaining a user identifier from a database rule table by the rule distribution management service, calculating a rule hash value of each rule according to the user identifier and a preset rule hash upper limit value, and storing the rule hash value into the database rule table;
the generation and storage unit is used for the rule distribution management service to generate a rule hash partition list which is required to be loaded by each rule processing service in a banding mode according to the preset hash partition rule, and stores the rule hash partition list to obtain rule configuration information;
the first sending unit is used for sending the rule configuration information to the corresponding rule processing service by the rule distribution management service and sending the total rule configuration information to a message dispatch service;
The first obtaining and loading unit is used for obtaining rules corresponding to the rule hash partition list from the database rule table according to the rule configuration information by the rule processing service for loading;
and the distribution unit is used for distributing the reported information according to the full rule configuration information and the user identification in the reported information if the reported information is received by the message dispatch service.
9. A mass rule processing system based on an internet of things platform, comprising a rule distribution management service, a rule processing service and a message dispatching service, wherein the rule distribution management service, the rule processing service and the message dispatching service each comprise a memory and a processor, the memory stores a computer program, and the processors of the rule distribution management service, the rule processing service and the message dispatching service implement the method of any one of claims 1-7 when executing the computer program.
10. A computer readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211644686.9A CN116303418A (en) | 2022-12-20 | 2022-12-20 | Mass rule processing method, system and storage medium based on Internet of things platform |
| PCT/CN2024/070262 WO2024131990A1 (en) | 2022-12-20 | 2024-01-03 | Internet of things platform-based mass rule processing method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211644686.9A CN116303418A (en) | 2022-12-20 | 2022-12-20 | Mass rule processing method, system and storage medium based on Internet of things platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116303418A true CN116303418A (en) | 2023-06-23 |
Family
ID=86802046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211644686.9A Pending CN116303418A (en) | 2022-12-20 | 2022-12-20 | Mass rule processing method, system and storage medium based on Internet of things platform |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116303418A (en) |
| WO (1) | WO2024131990A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024131990A1 (en) * | 2022-12-20 | 2024-06-27 | 天翼物联科技有限公司 | Internet of things platform-based mass rule processing method, system and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101299758B (en) * | 2008-05-21 | 2011-05-11 | 网御神州科技（北京）有限公司 | Well-regulated group system for cosmically processing event as well as processing method |
| CN107645403B (en) * | 2016-07-22 | 2020-07-03 | 阿里巴巴集团控股有限公司 | Terminal rule engine device and terminal rule operation method |
| CN108959562B (en) * | 2018-07-04 | 2020-06-30 | 北京京东尚科信息技术有限公司 | Massive regular data processing method and system applied to block chain |
| CN111639101B (en) * | 2020-04-27 | 2022-12-06 | 浙江时空道宇科技有限公司 | Method, device and system for correlating rule engine system of internet of things and storage medium |
| CN113384899B (en) * | 2021-07-05 | 2022-10-04 | 在线途游（北京）科技有限公司 | Rule-based operation method and system |
| CN115237595A (en) * | 2022-04-13 | 2022-10-25 | 杭州涂鸦信息技术有限公司 | Data processing method, data processing device, distribution server, data processing system, and storage medium |
| CN115145552A (en) * | 2022-05-23 | 2022-10-04 | 深圳市云恋科技有限公司 | Method, device and equipment for executing rule engine and computer readable storage medium |
| CN116303418A (en) * | 2022-12-20 | 2023-06-23 | 天翼物联科技有限公司 | Mass rule processing method, system and storage medium based on Internet of things platform |
-
2022
- 2022-12-20 CN CN202211644686.9A patent/CN116303418A/en active Pending
-
2024
- 2024-01-03 WO PCT/CN2024/070262 patent/WO2024131990A1/en unknown
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024131990A1 (en) * | 2022-12-20 | 2024-06-27 | 天翼物联科技有限公司 | Internet of things platform-based mass rule processing method, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024131990A1 (en) | 2024-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111522641B (en) | Task scheduling method, device, computer equipment and storage medium | |
| CN109032803B (en) | Data processing method and device and client | |
| CN111722918A (en) | Service identification code generation method and device, storage medium and electronic equipment | |
| CN113596094B (en) | Load balancing method and device based on cloud cluster | |
| CN106775470B (en) | Data storage method and system | |
| CN110888658A (en) | Dynamic method and device for functional module in application program and storage medium | |
| WO2024131990A1 (en) | Internet of things platform-based mass rule processing method, system and storage medium | |
| CN111988429A (en) | Algorithm scheduling method and system | |
| CN114461407B (en) | Data processing method, data processing device, distribution server, data processing system, and storage medium | |
| CN109756533B (en) | Mirror image acceleration method and device and server | |
| CN111522811A (en) | Database processing method and device, storage medium and terminal | |
| CN110300011B (en) | Alarm root cause positioning method, device and computer readable storage medium | |
| CN113377817A (en) | Data processing method, system, device and storage medium | |
| CN111049750B (en) | Message forwarding method, system and equipment | |
| CN116910061A (en) | Database splitting and table splitting method, device and equipment and readable storage medium | |
| CN113111351A (en) | Test method, test device and computer-readable storage medium | |
| CN116233254A (en) | Business cut-off method, device, computer equipment and storage medium | |
| CN110046040B (en) | Distributed task processing method and system and storage medium | |
| CN108173689B (en) | Output system of load balancing data | |
| CN115033551A (en) | Database migration method and device, electronic equipment and storage medium | |
| CN110083438B (en) | Transaction distribution method, device, equipment and storage medium | |
| CN108763498B (en) | User identity identification method and device, electronic equipment and readable storage medium | |
| CN111274209A (en) | Call bill file processing method and device | |
| CN110213314B (en) | Method, device and server for determining storage node | |
| CN117539649B (en) | Identification management method, equipment and readable storage medium of distributed cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |