CN116996599A - Data processing method, device, system, electronic equipment and computer storage medium - Google Patents

Data processing method, device, system, electronic equipment and computer storage medium Download PDF

Info

Publication number
CN116996599A
CN116996599A CN202310820066.4A CN202310820066A CN116996599A CN 116996599 A CN116996599 A CN 116996599A CN 202310820066 A CN202310820066 A CN 202310820066A CN 116996599 A CN116996599 A CN 116996599A
Authority
CN
China
Prior art keywords
data
data processing
adapter
feedback information
processing service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310820066.4A
Other languages
Chinese (zh)
Inventor
李喆
秦川
乔金良
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Siemens Cerberus Electronics Ltd
Original Assignee
Beijing Siemens Cerberus Electronics Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Siemens Cerberus Electronics Ltd filed Critical Beijing Siemens Cerberus Electronics Ltd
Priority to CN202310820066.4A priority Critical patent/CN116996599A/en
Publication of CN116996599A publication Critical patent/CN116996599A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/2866—Architectures; Arrangements
    • H04L67/2869—Terminals specially adapted for communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/26—Special purpose or proprietary protocols or architectures

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the application provides a data processing method, a device, a system, electronic equipment and a computer storage medium, wherein the method comprises the following steps: receiving first data sent by an adapter arranged at a data acquisition end; operating at least one data processing service created in advance according to the first data, and obtaining feedback information generated by the data processing service according to the first data; and sending the feedback information to the MQTT message server, and sending the feedback information to a data receiving end subscribed to the corresponding data processing service through the MQTT message server. In the data processing method of the scheme, the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard, so that the method has better universality, and the MQTT message server is used for sending feedback information to a data receiving end subscribed to corresponding data processing service, so that data intercommunication among different users can be realized. The pre-created data processing service can be flexibly configured, so that the method has better development capability.

Description

Data processing method, device, system, electronic equipment and computer storage medium
Technical Field
Embodiments of the present application relate to the field of data processing technologies, and in particular, to a data processing method, apparatus, system, electronic device, and computer storage medium.
Background
There are a large number of electromechanical devices in the building management system BMS (Building Management System), and related electromechanical devices and various applications adapted to these electromechanical devices can often generate billions of and even billions of mass small files, which brings great challenges to metadata management, access performance, storage efficiency and other aspects, and management problems of mass small files become industry-accepted problems.
In order to solve the management problem of massive small files, a set of semantic tag dictionary and method tool (Haystack) for describing building data is provided, and massive small files can be managed simply and conveniently through the Haystack.
However, the Haystack does not have a standardized data format, and the data description mode is generally defined by human beings, so that data intercommunication is difficult, and the running cost is increased.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application provide a data processing method, apparatus, system, electronic device, and computer storage medium, so as to at least solve or alleviate the above problems.
According to a first aspect of an embodiment of the present application, there is provided a data processing method, including: receiving first data sent by an adapter arranged at a data acquisition end, wherein the first data is acquired from the data acquisition end by the adapter through a Haystack interface, and the first data comprises operation data of Internet of things equipment; operating at least one data processing service created in advance according to the first data, and obtaining feedback information generated by the data processing service according to the first data; and sending the feedback information to an MQTT message server, and sending the feedback information to a data receiving end subscribing corresponding data processing service through the MQTT message server.
Optionally, the method may further include: transmitting authentication information to the data acquisition end through the adapter; after the data acquisition end responds to the authentication information to determine that the authentication is passed, a data request is sent to the adapter, so that the adapter acquires the first data from the data acquisition end according to the data request.
Optionally, the process of sending the data request to the adapter may include: and sending a data request to the adapter according to a preset polling period, so that the adapter periodically acquires the first data from the data acquisition end.
Optionally, the process of running at least one data processing service created in advance according to the first data and obtaining feedback information generated by the data processing service according to the first data may further include: extracting input data of the data processing service from the first data according to a preset data mapping relation; and if the input data of the data processing service is extracted from the first data, operating the data processing service based on the extracted input data, and obtaining feedback information generated by the data processing service according to the input data.
Optionally, the data processing service includes a data synchronization service or an alarm service; the data synchronization service is used for converting input data into output data meeting the MQTT communication protocol and determining the output data as the feedback information; the alarm service is used for generating alarm information when input data meet preset alarm conditions, and determining the alarm information as the feedback information.
Optionally, the method further comprises: and sending a data request to the adapter so that the adapter returns the first data according to the data request, wherein the first data is obtained after the adapter converts the operation data obtained from at least one point table into a character separation value format, the operation data included in the first data is data matched with a target tag indicated by the data request in the at least one point table, the point table is generated by the data acquisition end according to a tag of the operation data of the internet of things device, the tag of the operation data is added based on a Haystock system, and each point table comprises at least part of the operation data.
Optionally, the method further comprises: responding to a service creation request, creating a new data processing service, and generating a data mapping relation of the new data processing service, wherein the data mapping relation of the new data processing service is used for indicating required input data; and sending a service release request to the MQTT message server to create a subscription theme corresponding to the newly added data processing service on the MQTT message server.
According to a second aspect of an embodiment of the present application, there is provided a data processing apparatus including: the device comprises a receiving module, a processing module and a sending module; the receiving module is used for receiving first data sent by an adapter arranged at a data acquisition end, wherein the first data is acquired from the data acquisition end by the adapter through a Haystack interface, and the first data comprises operation data of Internet of things equipment; the processing module is used for operating at least one data processing service which is created in advance according to the first data, and obtaining feedback information generated by the data processing service according to the first data; the sending module is used for sending the feedback information to an MQTT message server, and sending the feedback information to a data receiving end subscribing corresponding data processing service through the MQTT message server.
According to a third aspect of an embodiment of the present application, there is provided a data processing system comprising: the device comprises a data acquisition end, an adapter, an MQTT message server and a data processing device; the data processing device is configured to perform the data processing method according to any one of the first aspects of the embodiments; the data acquisition end is used for acquiring operation data of the Internet of things equipment; the adapter is used for acquiring first data from the data acquisition end and sending the first data to the data processing device, wherein the first data comprises operation data of the Internet of things equipment; the MQTT message server is used for receiving feedback information sent by the data processing device and sending the feedback information to a data receiving end subscribing corresponding data processing service.
Optionally, the data acquisition end is configured to add a tag to the acquired operation data of the internet of things device based on the Haystack system, and generate at least one point table according to the tag of the operation data, where each point table includes at least part of the operation data; the adapter is used for receiving a data request from the data processing device, determining a target tag according to the data request, acquiring operation data matched with the target tag from each point table, and converting the acquired operation data into the first data in a character separation value format.
According to a fourth aspect of an embodiment of the present application, there is provided an electronic device including: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus; the memory is configured to store at least one executable instruction, where the executable instruction causes the processor to perform operations corresponding to the data processing method according to the first aspect of the foregoing embodiment.
According to a fifth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements a data processing method as in the first aspect of the embodiments described above.
According to a sixth aspect of embodiments of the present application, there is provided a computer program product comprising computer instructions for instructing a computing device to perform operations corresponding to the data processing method of the first aspect of the above embodiments.
According to the technical scheme, the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard (ISO/IEC PRF 20922), so that the method has good universality, and the MQTT message server is used for sending feedback information to a data receiving end subscribed to corresponding data processing service, so that data intercommunication among different users can be realized. The pre-created data processing service can be flexibly configured, so that the method has better development capability.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flow chart of a data processing method of one embodiment of the present application;
FIG. 2 is a flow chart of a data processing method according to another embodiment of the present application;
FIG. 3 is a schematic diagram of a data processing apparatus according to one embodiment of the present application;
FIG. 4 is a schematic diagram of a data processing system according to one embodiment of the present application;
fig. 5 is a schematic diagram of an electronic device according to an embodiment of the application.
List of reference numerals:
100: data processing method 200: the data processing method 300: data processing apparatus
301: the receiving module 302: processing module 303: transmitting module
400: data processing system 401: data acquisition end 402: adapter device
403: MQTT message server 50: the electronic device 51: processor and method for controlling the same
52: communication interface 53: a reservoir 54: communication bus
55: program
S101: receiving first data sent by an adapter arranged at a data acquisition end
S102: operating at least one data processing service created in advance according to the first data, and obtaining feedback information generated by the data processing service according to the first data
S103: the feedback information is sent to an MQTT message server, and the feedback information is sent to a data receiving end subscribing corresponding data processing service through the MQTT message server
S1021: extracting input data of the data processing service from the first data according to a preset data mapping relation
S1022: if the input data of the data processing service is extracted from the first data, the data processing service is operated based on the extracted input data, and feedback information generated by the data processing service according to the input data is obtained
Detailed Description
In order to better understand the technical solutions in the embodiments of the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the present application, shall fall within the scope of protection of the embodiments of the present application.
Data processing method
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present application, and as shown in FIG. 1, the data processing method 100 includes the steps of:
step S101, first data sent by an adapter arranged at a data acquisition end is received.
During the day, all data generated by electromechanical devices and related application software in the building management system can be recorded into hundreds of millions of DVDs, and the large data can be peeked. Along with the large-scale application of the internet of things technology, massive internet of things data, equipment, business and service respectively adopt different data description modes, so that the fragmentation problems of data isomerization, equipment isomerization, application isomerization and the like are caused, the development of industry is hindered, and therefore, the data needs to be processed. First data sent by an adapter arranged at a data acquisition end is received, and the data acquisition end can acquire and store operation data of the Internet of things equipment, for example, the operation data can be stored in a database of the data acquisition end. The received first data are acquired from the data acquisition end through the Haystack interface by the adapter, and the first data comprise operation data of the Internet of things equipment.
The data acquisition end can be a Haystack system, which can comprise a Haystack model, wherein the main structure of the Haystack model is a hierarchical structure based on three entities, in addition, other core entities can be defined by self, and part of data processing services in the pre-created data processing services can correspond to the other core entities. For example, the three entities of the Haystack model are hierarchically structured as Site, where a single building has its own street address, equip, physical or logical devices within the Site, and Point, sensors, actuators or settings of the devices. The data processing service may include monitoring an external temperature, where a Weather core entity may be defined in the Haystack model, and an external temperature tag may be marked on the external temperature data, where the first data sent by the adapter includes external temperature data of the external temperature tag. Based on the label mode, the manpower process of the traditional field engineering can be greatly reduced. For example, rule conditions are created in batches through label screening, and after filtering, the rule conditions are matched with actual operation objects, so that a great deal of repeated work of previous pure manual deployment is reduced.
And step S102, at least one data processing service which is created in advance is operated according to the first data, and feedback information generated by the data processing service according to the first data is obtained.
After the first data is obtained, at least one data processing service created in advance is operated according to the first data, for example, three data processing services are created in advance, two data processing services can be operated according to the first data, and feedback information generated by the two data processing services according to the first data is respectively obtained.
And step S103, the feedback information is sent to an MQTT message server, and the feedback information is sent to a data receiving end subscribing corresponding data processing services through the MQTT message server.
After obtaining the feedback information, the feedback information is sent to the MQTT message server. The data receiving end can obtain feedback information of the corresponding data processing service sent by the MQTT message server in a mode of subscribing the corresponding data processing service. The data receiving end can be an internet of things device, an internet of things platform or a third party device. The Internet of things equipment can acquire feedback information for realizing data synchronization through subscribing corresponding data processing services, the Internet of things platform can acquire feedback information for enabling the Internet of things platform to perform data synchronization on the Internet of things equipment, and the third party equipment can acquire feedback information for alarming through subscribing corresponding data processing services.
It should be noted that, the types of feedback information obtained by the internet of things device, the internet of things platform and the third party device are only used as examples, and are not limited, and in practical application, the data processing service may be flexibly configured to obtain feedback information required by service requirements and the like.
In the embodiment of the application, the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard (ISO/IEC PRF 20922), so that the method has better universality, and the data intercommunication among different users can be realized by sending feedback information to a data receiving end subscribed to corresponding data processing services through the MQTT message server. The pre-created data processing service can be flexibly configured, so that the method has better development capability.
In one possible implementation, the process of data processing may further include: the method comprises the steps that identity verification information is sent to a data acquisition end through an adapter, after the data acquisition end responds to the identity verification information to confirm that the identity verification is passed, a data request is sent to the adapter, and therefore the adapter acquires first data from the data acquisition end according to the data request.
In order to ensure the safety of data, the data acquisition end needs to perform data communication after performing identity verification, so that the data acquisition end sends identity verification information to the data acquisition end through the adapter, the data acquisition end performs identity verification, is disconnected from the adapter which does not pass the identity verification, sends a communication agreement message to the adapter which passes the identity verification, and can send a data request to the adapter so that the adapter can acquire first data from the data acquisition end according to the data request.
In the embodiment of the application, the adapter is used for sending the authentication information to the data acquisition end so that the data acquisition end can perform authentication, and the security of the data can be ensured.
In one possible implementation, the process of sending the data request to the adapter may further include: and sending a data request to the adapter according to a preset polling period so that the adapter periodically acquires the first data from the data acquisition end.
For example, the polling period may be set to 5S, and a data request is sent to the adapter every 5S, so that the adapter periodically acquires the first data from the data acquisition end.
In the embodiment of the application, the running data of the Internet of things equipment is updated continuously, and the running data of the updated Internet of things equipment can be acquired in time by sending the data request to the adapter in the polling period. In addition, the polling period can be set according to the update frequency of the operation data of the Internet of things equipment, so that resource waste caused by too short polling period is avoided.
In one possible implementation, the data processing service includes a data synchronization service or an alarm service; the data synchronization service is used for converting input data into output data meeting the MQTT communication protocol and determining the output data as feedback information; the alarm service is used for generating alarm information when the input data meets preset alarm conditions, and determining the alarm information as feedback information.
The data processing service may include a data synchronization service that converts input data, which may be first data or a part of the first data, into output data satisfying the MQTT communication protocol, determines the output data as feedback information, and transmits the feedback information to the MQTT message server. When the input data meets the preset alarm condition, the alarm service generates alarm information, for example, the preset alarm condition can be that the external environment temperature of the building exceeds 40 ℃, the data in the input data indicates that the external temperature of the building is 45 ℃, the alarm information can be defined by self, the alarm information is determined to be feedback information, and the feedback information is sent to the MQTT message server.
In the embodiment of the application, the data processing service comprises a data synchronization service and an alarm service, so that the updated operation data of the Internet of things equipment can be synchronized to the corresponding data receiving end, and the operation data of the Internet of things equipment can be timely pushed to the corresponding data receiving end when the operation data of the Internet of things equipment is abnormal or needs to be concerned.
In one possible implementation, the process of data processing may further include: and sending a data request to the adapter so that the adapter returns the first data according to the data request.
The first data is obtained by converting operation data obtained from at least one point table into a character separation value format by an adapter, the operation data included in the first data is data matched with a target label indicated by a data request in at least one point table, the point table is generated by a data acquisition end according to the label of the operation data of the Internet of things device, the label of the operation data is added based on a Haystock system, and each point table comprises at least part of operation data.
For example, the Haystack system can name tags of running data of the Internet of things equipment according to Camel naming method (Camel-Case), and the names of variables and functions are formed by mixing letters in Case and Case, so that the readability of the tags is good.
After the tag is added to the operation data of the internet of things device, the data acquisition end generates at least one Point table according to the tag of the operation data, for example, the hierarchical structures of different Point tables can be different, site can be generated, wherein a single building has a street address of the single building, point is a physical or logical device in a Site, and Point is a three-level structure of a sensor of the device, and the Point tables of other core entities can be defined by the user. Each point table includes at least a portion of the operational data.
After receiving the data request, the adapter determines a target tag according to the data request, obtains running data matched with the target tag from each point table in a tag searching mode, converts the running data into first data in a common-Separated Values (CSV) format, and stores table data in a plain text form. For example, the character separation value format may be comma separation value format, separating different data by comma.
In the embodiment of the application, the tag is added to the collected operation data of the Internet of things equipment based on the Haystack system, so that the data can be managed, the corresponding data can be conveniently obtained according to the target tag, and the data processing efficiency is improved.
In one possible implementation, the process of data processing may further include: in response to the service creation request, creating a new data processing service, and generating a data mapping relation of the new data processing service, and then sending a service release request to the MQTT message server to create a subscription theme corresponding to the new data processing service on the MQTT message server.
In order to meet the diversity of the data processing services, a service creation request can be written according to the service requirement or the actual requirement, so as to create a new data processing service according to the service creation request, and meanwhile, in order to ensure that the new data processing service operates normally, a data mapping relation of the new data processing service is generated, wherein the data mapping relation of the new data processing service is used for indicating required input data. After the newly added data processing service is created, a service release request needs to be synchronously sent to the MQTT message server so that the MQTT message server creates a subscription theme corresponding to the newly added data processing service, and a data receiving end can acquire feedback information corresponding to the newly added data processing service in a manner of subscribing the subscription theme of the newly added data processing service.
In the embodiment of the application, the newly added data processing service can be created according to the requirement through the service creation request, so that the diversity of data processing is improved, and the method is suitable for more diversified scenes and actual service requirements.
Fig. 2 is a flowchart of a data processing method according to another embodiment of the present application, and as shown in fig. 2, in the data processing method 200, step S102 may include the following sub-steps:
step S1021, extracting the input data of the data processing service from the first data according to a preset data mapping relation.
The specific processing mode of the first data may be that firstly, input data of the data processing service is extracted from the first data according to a preset data mapping relation, and the input data is data required by the operation of the data processing service.
Step S1022, if the input data of the data processing service is extracted from the first data, the data processing service is operated based on the extracted input data, and feedback information generated by the data processing service according to the input data is obtained.
For example, the data processing service needs operation voltage data of the internet of things device, if the operation voltage data of the internet of things device exists in the first data and is extracted, the data processing service is operated based on the extracted operation voltage data of the internet of things device, and feedback information generated by the data processing service according to the operation voltage data of the internet of things device is obtained.
In the embodiment of the application, the data processing service is extracted from the first data through the preset data mapping relation, so that the accuracy of the extraction of the input data can be improved, and the efficiency of data processing is further improved.
Data processing apparatus
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present application, and as shown in FIG. 3, the data processing apparatus 300 includes: a receiving module 301, a processing module 302 and a transmitting module 303.
The receiving module 301 is configured to receive first data sent by an adapter disposed at a data collection end, where the first data is obtained by the adapter from the data collection end through a Haystack interface, and the first data includes operation data of an Internet of things device.
First, the receiving module 301 receives first data sent by an adapter disposed at a data collection end, where the data collection end may collect and store operation data of the internet of things device, for example, the operation data may be stored in a database at the data collection end. The received first data are acquired from the data acquisition end through the Haystack interface by the adapter, and the first data comprise operation data of the Internet of things equipment.
The processing module 302 is configured to operate at least one data processing service created in advance according to the first data, and obtain feedback information generated by the data processing service according to the first data.
After the receiving module 301 obtains the first data, the processing module 302 executes at least one data processing service created in advance according to the first data, for example, three data processing services are created in advance, and two of the data processing services may be executed according to the first data, so as to obtain feedback information generated by the two data processing services according to the first data.
And the sending module 303 is configured to send the feedback information to the MQTT message server, and send the feedback information to a data receiving end subscribed to the corresponding data processing service through the MQTT message server.
After the processing module 302 obtains the feedback information, the sending module 303 sends the feedback information to the MQTT message server. The data receiving end can obtain feedback information of the corresponding data processing service sent by the MQTT message server in a mode of subscribing the corresponding data processing service. The data receiving end can be an internet of things device, an internet of things platform or a third party device. The Internet of things equipment can acquire feedback information for realizing data synchronization through subscribing corresponding data processing services, the Internet of things platform can acquire feedback information for enabling the Internet of things platform to perform data synchronization on the Internet of things equipment, and the third party equipment can acquire feedback information for alarming through subscribing corresponding data processing services.
It should be noted that, the types of feedback information obtained by the internet of things device, the internet of things platform and the third party device are only used as examples, and are not limited, and in practical application, the data processing service may be flexibly configured to obtain feedback information required by service requirements and the like.
In the embodiment of the application, the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard (ISO/IEC PRF 20922), so that the method has better universality, and the data intercommunication among different users can be realized by sending feedback information to a data receiving end subscribed to corresponding data processing services through the MQTT message server. The pre-created data processing service can be flexibly configured, so that the method has better development capability.
Data processing system
FIG. 4 is a schematic diagram of a data processing system according to one embodiment of the application. As shown in fig. 4, the data processing system 400 includes: a data acquisition end 401, an adapter 402, an MQTT message server 403, and a data processing device 300.
The data processing device 300 is used to perform the relevant steps of the data processing method embodiments described above.
The data acquisition end 401 is configured to acquire operation data of the internet of things device.
The data collection terminal 401 may collect and store the operation data of the internet of things device, for example, the operation data may be stored in a database of the data collection terminal 401.
The adapter 402 is configured to obtain first data from the data collection end 401, and send the first data to the data processing apparatus 300, where the first data includes operation data of the internet of things device.
The adapter 402 obtains the first data from the data acquisition terminal 401 through a Haystack interface.
The MQTT message server 403 is configured to receive feedback information sent by the data processing apparatus 300, and send the feedback information to a data receiving end subscribed to a corresponding data processing service.
The data receiving end can obtain feedback information of the corresponding data processing service sent by the MQTT message server 403 by subscribing to the corresponding data processing service.
In the embodiment of the application, the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard (ISO/IEC PRF 20922), so that the method has better universality, and the MQTT message server 403 sends feedback information to a data receiving end subscribed to corresponding data processing service, so that data intercommunication among different users can be realized. The pre-created data processing service can be flexibly configured, so that the method has better development capability.
In a possible implementation manner, the data collection end 401 is configured to add a tag to collected operation data of the internet of things device based on the Haystack system, and generate at least one point table according to the tag of the operation data, where each point table includes at least part of the operation data.
For example, the Haystack system can name tags of running data of the Internet of things equipment according to Camel naming method (Camel-Case), and the names of variables and functions are formed by mixing letters in Case and Case, so that the readability of the tags is good.
After adding a tag to the operation data of the internet of things device, the data collection end 401 generates at least one Point table according to the tag of the operation data, for example, the hierarchical structures of different Point tables may be different, and Site may be generated, where a single building has its own street address, equipment is physical or logical devices in a Site, and Point is a three-hierarchical structure of sensors of the devices, and the Point tables of other core entities may also be defined by itself. Each point table includes at least a portion of the operational data.
The adapter 402 is configured to receive a data request from the data processing apparatus 300, determine a target tag according to the data request, obtain operation data matching the target tag from the table of points, and convert the obtained operation data into first data in the format of character separation values.
After receiving the data request from the data processing apparatus 300, the adapter 402 determines the target tag according to the data request, and then obtains the running data matched with the target tag from the point table by means of tag lookup, and converts the running data into first data in a common-Separated Values (CSV) format, and stores the table data in a plain text form. For example, the character separation value format may be comma separation value format, separating different data by comma.
In the embodiment of the application, the data acquisition end 401 adds a tag to the acquired operation data of the internet of things equipment based on the Haystack system, so that the data can be managed, the adapter 402 can acquire corresponding data according to the target tag conveniently, and the data processing efficiency is improved.
Electronic equipment
Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present application, and the specific embodiment of the present application is not limited to the specific implementation of the electronic device. As shown in fig. 5, the electronic device 50 may include: a processor (processor) 51, a communication interface (Communications Interface) 52, a memory (memory) 53, and a communication bus 54. Wherein:
processor 51, communication interface 52, and memory 53 communicate with each other via communication bus 54.
Communication interface 52 for communicating with other electronic devices or servers.
The processor 51 is configured to execute the program 55, and may specifically perform relevant steps in the foregoing data processing method embodiment.
In particular, program 55 may include program code including computer-operating instructions.
The processor 51 may be a CPU or a specific integrated circuit ASIC (Application Specific Integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors comprised by the smart device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
A memory 53 for storing a program 55. The memory 53 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 55 may be used in particular to cause the processor 51 to execute the data processing method in the foregoing embodiment.
The specific implementation of each step in the program 55 may refer to the corresponding steps and corresponding descriptions in the units in the foregoing data processing method embodiment, which are not repeated herein. It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and modules described above may refer to corresponding procedure descriptions in the foregoing method embodiments, which are not repeated herein.
The electronic equipment of the embodiment of the application has better universality because the MQTT is a message protocol based on a publish/subscribe paradigm under the ISO standard (ISO/IEC PRF 20922), and can realize the data intercommunication among different users by sending feedback information to a data receiving end subscribed to corresponding data processing service through the MQTT message server. The pre-created data processing service can be flexibly configured, so that the method has better development capability.
Computer storage medium
The present application also provides a computer readable storage medium storing instructions for causing a machine to perform a data processing method as described herein. Specifically, a system or apparatus provided with a storage medium on which a software program code realizing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or apparatus may be caused to read out and execute the program code stored in the storage medium.
In this case, the program code itself read from the storage medium may realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code form part of the present application.
Examples of the storage medium for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer by a communication network.
Computer program product
Embodiments of the present application also provide a computer program product comprising computer instructions that instruct a computing device to perform any corresponding operations of the above-described method embodiments.
It should be noted that, according to implementation requirements, each component/step described in the embodiments of the present application may be split into more components/steps, or two or more components/steps or part of operations of the components/steps may be combined into new components/steps, so as to achieve the objects of the embodiments of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored on such software processes on a recording medium using a general purpose computer, special purpose processor, or programmable or special purpose hardware such as an ASIC or FPGA. It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by a computer, processor, or hardware, performs the methods described herein. Furthermore, when a general purpose computer accesses code for implementing the methods illustrated herein, execution of the code converts the general purpose computer into a special purpose computer for performing the methods illustrated herein.
It should be noted that not all the steps and modules in the above flowcharts and the system configuration diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution sequence of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented jointly by some components in multiple independent devices.
Nouns and pronouns for humans in this patent application are not limited to a particular gender.
In the above embodiments, the hardware module may be mechanically or electrically implemented. For example, a hardware module may include permanently dedicated circuitry or logic (e.g., a dedicated processor, FPGA, or ASIC) to perform the corresponding operations. The hardware modules may also include programmable logic or circuitry (e.g., a general-purpose processor or other programmable processor) that may be temporarily configured by software to perform the corresponding operations. The particular implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.
While the application has been illustrated and described in detail in the drawings and in the preferred embodiments, the application is not limited to the disclosed embodiments, and those skilled in the art will appreciate that many more embodiments of the application can be obtained by combining the code audits in the different embodiments and still fall within the scope of the application.

Claims (13)

1. A data processing method (100), comprising:
receiving first data sent by an adapter arranged at a data acquisition end, wherein the first data is acquired from the data acquisition end by the adapter through a Haystack interface, and the first data comprises operation data of Internet of things equipment;
operating at least one data processing service created in advance according to the first data, and obtaining feedback information generated by the data processing service according to the first data;
and sending the feedback information to an MQTT message server, and sending the feedback information to a data receiving end subscribing corresponding data processing service through the MQTT message server.
2. The method of claim 1, wherein the method further comprises:
transmitting authentication information to the data acquisition end through the adapter;
after the data acquisition end responds to the authentication information to determine that the authentication is passed, a data request is sent to the adapter, so that the adapter acquires the first data from the data acquisition end according to the data request.
3. The method of claim 2, wherein the sending a data request to the adapter comprises:
and sending a data request to the adapter according to a preset polling period, so that the adapter periodically acquires the first data from the data acquisition end.
4. The method of claim 1, wherein the running at least one pre-created data processing service from the first data, obtaining feedback information generated by the data processing service from the first data, comprises:
extracting input data of the data processing service from the first data according to a preset data mapping relation;
and if the input data of the data processing service is extracted from the first data, operating the data processing service based on the extracted input data, and obtaining feedback information generated by the data processing service according to the input data.
5. The method of claim 1, wherein the data processing service comprises a data synchronization service or an alarm service;
the data synchronization service is used for converting input data into output data meeting the MQTT communication protocol and determining the output data as the feedback information;
the alarm service is used for generating alarm information when input data meet preset alarm conditions, and determining the alarm information as the feedback information.
6. The method of claim 1, the method further comprising: and sending a data request to the adapter so that the adapter returns the first data according to the data request, wherein the first data is obtained after the adapter converts the operation data obtained from at least one point table into a character separation value format, the operation data included in the first data is data matched with a target tag indicated by the data request in the at least one point table, the point table is generated by the data acquisition end according to a tag of the operation data of the internet of things device, the tag of the operation data is added based on a Haystock system, and each point table comprises at least part of the operation data.
7. The method of any one of claims 1-6, the method further comprising:
responding to a service creation request, creating a new data processing service, and generating a data mapping relation of the new data processing service, wherein the data mapping relation of the new data processing service is used for indicating required input data;
and sending a service release request to the MQTT message server to create a subscription theme corresponding to the newly added data processing service on the MQTT message server.
8. A data processing apparatus (300), comprising: a receiving module (301), a processing module (302) and a transmitting module (303);
the receiving module (301) is configured to receive first data sent by an adapter disposed at a data acquisition end, where the first data is acquired by the adapter from the data acquisition end through a Haystack interface, and the first data includes operation data of an internet of things device;
the processing module (302) is configured to operate at least one data processing service created in advance according to the first data, and obtain feedback information generated by the data processing service according to the first data;
the sending module (303) is configured to send the feedback information to an MQTT message server, and send the feedback information to a data receiving end subscribed to a corresponding data processing service through the MQTT message server.
9. A data processing system (400), comprising: a data acquisition end (401), an adapter (402), an MQTT message server (403) and a data processing device (300);
said data processing device (300) being adapted to perform the data processing method of any of claims 1-6;
the data acquisition end (401) is used for acquiring operation data of the Internet of things equipment;
the adapter (402) is configured to acquire first data from the data acquisition end (401) and send the first data to the data processing device (300), where the first data includes operation data of an internet of things device;
the MQTT message server (403) is configured to receive feedback information sent by the data processing device (300), and send the feedback information to a data receiving end subscribed to a corresponding data processing service.
10. The system of claim 9, wherein,
the data acquisition end (401) is used for adding a label to the acquired operation data of the Internet of things equipment based on the Haystack system, and generating at least one point table according to the label of the operation data, wherein each point table comprises at least part of the operation data;
the adapter (402) is configured to receive a data request from the data processing apparatus (300), determine a target tag according to the data request, acquire operation data matched with the target tag from each point table, and convert the acquired operation data into the first data in a character separation value format.
11. An electronic device (50), comprising: the processor (51), the communication interface (52), the memory (53) and the communication bus (54), and the processor (51), the memory (53) and the communication interface (52) complete communication with each other through the communication bus (54);
the memory (53) is configured to store at least one executable instruction that causes the processor (51) to perform operations corresponding to the method according to any one of claims 1-7.
12. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1-7.
13. A computer program product comprising computer instructions that instruct a computing device to perform operations corresponding to the method of any one of claims 1-7.
CN202310820066.4A 2023-07-05 2023-07-05 Data processing method, device, system, electronic equipment and computer storage medium Pending CN116996599A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310820066.4A CN116996599A (en) 2023-07-05 2023-07-05 Data processing method, device, system, electronic equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310820066.4A CN116996599A (en) 2023-07-05 2023-07-05 Data processing method, device, system, electronic equipment and computer storage medium

Publications (1)

Publication Number Publication Date
CN116996599A true CN116996599A (en) 2023-11-03

Family

ID=88529257

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310820066.4A Pending CN116996599A (en) 2023-07-05 2023-07-05 Data processing method, device, system, electronic equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN116996599A (en)

Similar Documents

Publication Publication Date Title
EP3709227B1 (en) System and method for interoperable communication of an automation system component with multiple information sources
US8966031B2 (en) Method for addressing management object in management tree and associated device management system
CN111683066B (en) Heterogeneous system integration method, heterogeneous system integration device, computer equipment and storage medium
CN107800565B (en) Inspection method, inspection device, inspection system, computer equipment and storage medium
WO2004046963A1 (en) Method and device for defining objects allowing to establish a device management tree for mobile communication devices
CN114385759B (en) Configuration file synchronization method and device, computer equipment and storage medium
CN110932918B (en) Log data acquisition method and device and storage medium
US10296515B2 (en) Engineering data collection from control and instrumentation systems
EP2933726B1 (en) Apparatus, system and method for application log data processing
CN112559813B (en) Internet of things gateway data processing method and device based on instruction association pushing
WO2018202440A1 (en) Data transmission method and apparatus
US9781010B2 (en) Managing remote devices
US20140136620A1 (en) Protocol conversion apparatus and method
CN116996599A (en) Data processing method, device, system, electronic equipment and computer storage medium
US11288158B2 (en) Device identification device and device identification method
CN111880949B (en) Service information processing method, system, device, server and terminal
CN112491596B (en) Cloud-based fault processing method and device
CN106993036B (en) Running environment information processing method and switch
CN112801136B (en) Internet of things gateway data processing method and device with characteristic identification
CN109193592B (en) Relay protection data transmission method between relay protection information systems
CN115314572B (en) Protocol conversion method, system and equipment for power communication protocol
CN112202905B (en) Information storage method, device and equipment and computer readable storage medium
CN116028444B (en) File fingerprint generation method, device and system, electronic equipment and storage medium
CN114143018B (en) Intelligent operation platform information encryption method and system
US20230289354A1 (en) Endpoint scan and profile generation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination