CN115665287A - Data processing method, analysis service and Internet of things platform - Google Patents

Data processing method, analysis service and Internet of things platform Download PDF

Info

Publication number
CN115665287A
CN115665287A CN202211304440.7A CN202211304440A CN115665287A CN 115665287 A CN115665287 A CN 115665287A CN 202211304440 A CN202211304440 A CN 202211304440A CN 115665287 A CN115665287 A CN 115665287A
Authority
CN
China
Prior art keywords
protocol
data packet
target
data
identifier
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211304440.7A
Other languages
Chinese (zh)
Inventor
尹相宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gree Electric Appliances Inc of Zhuhai
Original Assignee
Gree Electric Appliances Inc of Zhuhai
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gree Electric Appliances Inc of Zhuhai filed Critical Gree Electric Appliances Inc of Zhuhai
Priority to CN202211304440.7A priority Critical patent/CN115665287A/en
Publication of CN115665287A publication Critical patent/CN115665287A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)
  • Communication Control (AREA)

Abstract

The application provides a data processing method, an analytic service and an Internet of things platform. The processing method comprises the following steps: determining a target protocol of a data packet under the condition that the received data packet sent by the target gateway is an uplink data packet; determining a corresponding target coding and decoding processor at least according to a target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet; and performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data. According to the processing method, under the condition that the communication protocol and the data protocol corresponding to the data packet are known, the corresponding coding and decoding processor can be determined so as to realize format conversion of the data packet, so that the processing method can process any type of data packet, data can be processed more flexibly and reliably, and the problem that the flexibility of data processing is poor in the prior art is solved.

Description

Data processing method, analysis service and Internet of things platform
Technical Field
The application relates to the technical field of the Internet of things, in particular to a data processing method, an analysis service, a computer readable storage medium and an Internet of things platform.
Background
Currently, a large number of IoT (Internet of Things, ioT for short) terminals access to an Internet of Things platform through different communication modules, that is, the IoT terminals realize communication with a server (a server corresponding to the Internet of Things platform) through the communication modules, so as to realize processing of related services and realization of related functions.
Because the communication protocols supported by different communication modules are different and the data protocols supported by different IoT terminals are different, the collection and conversion of the internet of things data need to be oriented to different types of communication protocols and IoT terminals. Meanwhile, with the development of the related technology of the internet of things and the increase of application scenes, the types of communication protocols required to be supported by the internet of things platform are continuously increased, and frequent changes of the communication protocols are faced in some use scenes, which puts higher requirements on the reliability and flexibility of the communication data processing of the server and the convenience of use. The traditional protocol access and analysis mode of the server cannot meet the current requirements on the reliability and flexibility of data processing.
Therefore, a method for processing data more flexibly is needed.
Disclosure of Invention
The application mainly aims to provide a data processing method, an analysis service, a computer readable storage medium and an internet of things platform so as to solve the problem that the flexibility of data processing is poor in the prior art.
According to an aspect of the embodiments of the present invention, there is provided a data processing method applied in a resolution service, where the resolution service communicates with a plurality of target gateways, one target gateway communicates with at least one communication module, and one communication module connects with at least one IoT terminal, the data processing method includes: determining a target protocol of the data packet under the condition that the received data packet sent by the target gateway is an uplink data packet, wherein the target protocol comprises at least one of the following protocols: a communication protocol, a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal; determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet; and performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data.
Optionally, the determining, by the module identifier, a target protocol of the data packet when the received data packet sent by the target gateway is an uplink data packet includes: determining the target protocol based on the channel unique identifier of the data packet under the condition that the corresponding target protocol can be determined based on the channel unique identifier of the data packet; and under the condition that the corresponding target protocol cannot be determined based on the channel unique identifier of the data packet, determining the target protocol of the data packet at least according to the module identifier carried by the data packet.
Optionally, determining the target protocol of the data packet at least according to the module identifier carried by the data packet includes: determining the communication protocol according to the module identification of the data packet; searching a corresponding product identifier in a communication module table according to the module identifier of the data packet, wherein the communication module table at least comprises the module identifier and the product identifier, and the product identifier is an identifier of a product formed by the communication protocol and the data protocol; and determining the data protocol of the corresponding data packet according to the communication protocol and the product identification.
Optionally, determining a corresponding target codec processor according to at least the target protocol of the data packet includes: under the condition that an encoding and decoding processor corresponding to the data packet is found in a protocol item table according to a protocol identifier, determining the found encoding and decoding processor as the corresponding target encoding and decoding processor, wherein the protocol identifier is the identifier of the target protocol of the data packet, and the protocol item table at least comprises the protocol identifier and the encoding and decoding processor corresponding to the protocol identifier; and under the condition that the coding and decoding processor corresponding to the data packet is not found in the protocol item table according to the protocol identifier, determining a preset coding and decoding processor as the corresponding target coding and decoding processor.
Optionally, the processing method further includes: and determining a preset coding and decoding processor as the corresponding target coding and decoding processor under the condition that the target protocol of the data packet is not determined at least according to the module identification of the target gateway carried by the data packet.
Optionally, performing format conversion on the data packet at least according to the target codec processor to obtain target data, including: and according to the target coding and decoding processor and the corresponding data item of the target protocol, carrying out format conversion on the corresponding data packet to obtain the target data, wherein the data item is used for marking the position of the transmission value of the communication protocol or the data protocol.
Optionally, after performing format conversion on the corresponding data packet according to the target codec processor and the corresponding data item of the target protocol to obtain the target data, the processing method further includes: determining whether a sub-protocol identifier exists in the data packet, wherein the sub-protocol identifier is the protocol identifier of a sub-protocol, and the sub-protocol is the communication protocol or the data protocol; determining whether to find out the corresponding coding and decoding processor in a protocol item table based on the sub-protocol identifier under the condition that the sub-protocol identifier exists in the data packet, wherein the protocol item table at least comprises the protocol identifier and the coding and decoding processor corresponding to the protocol identifier; determining the coding and decoding processor as the target coding and decoding processor of the corresponding data packet under the condition that the corresponding coding and decoding processor is found in the protocol item table based on the sub-protocol identification, and determining a preset coding and decoding processor as the target coding and decoding processor of the corresponding data packet under the condition that the corresponding coding and decoding processor is not found in the protocol item table based on the sub-protocol identification; and carrying out format conversion on the data packet based on the target coding and decoding processor to obtain the corresponding target data.
According to another aspect of the embodiments of the present invention, there is also provided a resolution service, the resolution service communicating with a plurality of target gateways, one of the target gateways communicating with at least one communication module, one of the communication modules connecting with at least one IoT terminal, the resolution service including: a first determining unit, configured to determine a target protocol of a received data packet sent by the target gateway, where the received data packet is an uplink data packet, where the target protocol includes at least one of: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal; a second determining unit, configured to determine a corresponding target codec processor according to at least the target protocol of the data packet, where the target codec processor is configured to perform format conversion on the corresponding data packet; and the first conversion unit is used for performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium including a stored program, wherein the program executes any one of the data processing methods.
According to another aspect of the embodiments of the present invention, there is also provided an internet of things platform, including: the access service comprises a plurality of target gateways, and the target gateways are used for sending the received data packets sent by the IoT terminals to the resolution service; the analysis service is communicated with the access service, and the analysis service is used for executing any data processing method; and the business service is communicated with the analysis service and is used for acquiring the target data converted by the analysis service.
In the embodiment of the present invention, in the data processing method, first, after receiving a data packet sent by a target gateway, it is determined whether the received data packet is an uplink data packet, and in a case that the data packet is the uplink data packet, a target protocol of the data packet is determined, that is, a communication protocol and a data protocol of the data packet are determined; then, determining a corresponding target coding and decoding processor at least according to a target protocol of the data packet, namely the communication protocol and/or the data protocol; and finally, performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by a service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 shows a flow diagram of a method of processing data of an embodiment of the present application;
FIG. 2 shows a schematic diagram of a packet structure according to an embodiment of the present application;
FIG. 3 illustrates a schematic diagram of a communication protocol payload of an embodiment of the present application;
FIG. 4 illustrates a schematic diagram of a communication protocol payload of another embodiment of the present application;
FIG. 5 shows a schematic diagram of a parsing service of an embodiment of the present application;
fig. 6 shows an architecture diagram of an internet of things platform according to an embodiment of the present application;
FIG. 7 shows a flow diagram of a processing scheme for data of an embodiment of the present application.
Wherein the figures include the following reference numerals:
100. a data packet; 101. a communication protocol header; 102. a data protocol portion; 103. verifying the tail; 104. a communication protocol payload; 105. a first data protocol; 106. first transmission data; 107. second transmission data; 108. an embedded subprotocol; 109. sub-protocol data; 110. transmitting the verification tail; 111. a first data payload; 112. a second data protocol; 113. a second data payload; 114. an IoT terminal; 115. a communication module; 116. accessing a service; 117. resolving service; 118. service; 119. and (4) a target gateway.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances in order to facilitate the description of the embodiments of the application herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of description, some terms or expressions referred to in the embodiments of the present application are explained below:
communication protocol (Communications protocol): refers to the rules and conventions that must be followed by two entities to complete a communication or service. In computer communication, a communication protocol is used for realizing a standard between a computer and a network connection, and if the network does not have a uniform communication protocol, information transmission between the computers cannot be identified. The communication protocol refers to a communication rule predetermined in advance by the communicating parties.
Data protocol: refers to the format of data for both entities to complete data interaction.
As mentioned in the background of the invention, in order to solve the above problem, the prior art has poor flexibility in data processing, and in an exemplary embodiment of the present application, a data processing method, an analysis service, a computer-readable storage medium, and an internet of things platform are provided.
According to an embodiment of the present application, a method of processing data is provided.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application. The processing method is applied in a resolution service (a server), the resolution service communicates with a plurality of target gateways, one target gateway communicates with at least one communication module, and one communication module connects to at least one IoT terminal, as shown in fig. 1, the processing method includes the following steps:
step S101, determining a target protocol of the data packet when the received data packet sent by the target gateway is an uplink data packet, where the target protocol includes at least one of the following: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
step S102, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet;
step S103, at least according to the target coding and decoding processor, carrying out format conversion on the data packet to obtain target data.
In the above data processing method, first, after receiving a data packet sent by a target gateway, it is determined whether the received data packet is an uplink data packet, and if the data packet is the uplink data packet, a target protocol of the data packet, that is, a communication protocol and a data protocol of the data packet, is determined; then, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, namely the communication protocol and/or the data protocol; and finally, performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by a service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
Specifically, the data transmission process of the uplink data packet (which may also be referred to as an upload data packet) is as follows: the IoT terminal sends an initial data packet (the initial data packet does not carry a communication protocol at the moment) to the communication module; the communication module carries out protocol encapsulation on the received initial data packet based on a communication protocol of the communication module to obtain a preset data packet, and sends the preset data packet to a corresponding target gateway. And when the target gateway receives a preset data packet sent by the communication module, adding a channel unique identifier and a module identifier to the preset data packet to obtain the data packet. The target gateway puts the data packet into a message queue (such as an MQ message queue), and waits until the analysis service consumption is reached, so as to perform format conversion on the data packet to obtain target data. In an actual application process, the analytic service is sent to the target gateway, and then the target gateway sends the analytic service to the communication module, so that a data packet sent to the IoT terminal may be referred to as a downlink data packet (or may also be referred to as an issued data packet).
Specifically, the data packet may also be a data packet obtained after the destination gateway performs packet processing on the data stream or the datagram sent by the communication module. After the target gateway receives the data packet, the target gateway sends the data packet to the resolution service. Under the condition that the target gateway has the identity authentication function, the target gateway can also perform identity authentication on a data packet, so that the data packet subsequently sent to the analysis service can carry a channel unique identifier (namely, a global unique identifier of the Socket). Of course, the unique identifier of the gateway message, the data type of the data packet, the original data encoding format, the communication data (packet/frame), the communication time, and the extension data (for example, the IP (Internet Protocol Address, IP for short) Address of the source of the data packet, the service level, the will order, etc.) may also be carried.
In a specific embodiment of the present application, the number of target codec processors for performing format conversion on one data packet may be one, or may be multiple. That is, when the format of the packet is converted, the number of target codec processors used may be determined according to the data of the valid data portion in the packet. For example, for a data packet, in the valid data of the data packet, the first byte carries a temperature value, a target codec processor may be used to perform format conversion on the first byte to obtain a format-converted temperature value; if the second byte to the fifth byte in the valid data carry failure information, another target codec processor may be used to perform format conversion on the second byte to the fifth byte, so as to obtain failure information after format conversion. Of course, in practical application, the target codec processor may also be adjusted and set according to the data type, coding format, and the like of the valid data in the data packet, and is not limited to the target codec processor mentioned in the above example.
In an actual application process, a communication protocol is added to the received initial data packet of the IoT terminal at the communication module, so as to obtain a preset data packet. After the communication module sends the preset data packet to the target gateway in the form of data stream or datagram, the target gateway needs to perform packet-packet processing on the received data stream or datagram to convert the data stream or datagram into an independent data packet. For a packet, it is essentially a separate packet or frame of data. Which may carry a data packet header, a data packet payload, and a data packet checksum or data packet trailer, among other things. In the processing method, the protocol used by the data packet is divided into a communication protocol and a data protocol, so that the data packet received by the target gateway is the communication protocol of unlimited level and the nesting of the data protocol. Specifically, as shown in fig. 2, for example, a data packet 100 includes a communication protocol header 101 and a communication protocol payload 104, and the communication protocol payload 104 includes an embedded data protocol portion 102 and a check tail 103. As shown in fig. 3, the communication protocol payload 104 may further include a first data protocol 105, a first data payload 111, and a transmission check tail 110 (in an actual application, the communication protocol payload 104 may further not include the transmission check tail 110), and for the first data payload 111, the first transmission data 106 and the second transmission data 107 may be embedded inside, and the sub-protocol 108 and the sub-protocol data 109 (i.e., a valid data portion of the sub-protocol) may be embedded inside. As shown in fig. 4, the communication protocol payload 104 may further include a second data protocol 112 and a second data payload 113, wherein the second data payload 113 may further embed the plurality of transmission data.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than here.
In an embodiment of the present application, the determining a target protocol of the data packet when the received data packet sent by the target gateway is an uplink data packet includes: determining the target protocol based on the channel unique identifier of the data packet, if the corresponding target protocol can be determined based on the channel unique identifier of the data packet; and under the condition that the corresponding target protocol cannot be determined based on the channel unique identifier of the data packet, determining the target protocol of the data packet at least according to the module identifier carried by the data packet, so that the analysis service can determine the communication protocol and the data protocol of the data packet relatively simply, and the calculation amount of the analysis service is relatively small.
In order to determine a communication protocol and a data protocol used by a data packet more simply, in another embodiment of the present application, determining the target protocol of the data packet at least according to the module identifier carried by the data packet includes: determining the communication protocol according to the module identifier of the data packet; searching a corresponding product identifier in a communication module table according to the module identifier of the data packet, wherein the communication module table at least comprises the module identifier and a product identifier, and the product identifier is an identifier of a product formed by the communication protocol and the data protocol; and determining the data protocol of the corresponding data packet according to the communication protocol and the product identifier. That is to say, according to the module identifier of the target gateway, the outer layer communication protocol of the data packet can be obtained, then the product identifier corresponding to the communication protocol is obtained based on the pre-stored communication module table, the data protocol corresponding to the data packet is obtained, and the data protocol and the communication protocol corresponding to the data packet are further determined relatively simply and accurately.
Specifically, in the actual application process, the communication module table is preset in the parsing service. The communication module table has a module identification column, a product identification column and a communication protocol identification column. Wherein the product identification column has a plurality of products therein. The products are composed of different communication protocols and data protocols. In a specific embodiment of the present application, communication protocol a and data protocol a constitute one product, communication protocol a and data protocol B constitute another product, and so on.
In order to determine a target codec processor corresponding to a data packet more simply, in another embodiment of the present application, determining a corresponding target codec processor at least according to the target protocol of the data packet includes: under the condition that an encoding and decoding processor corresponding to the data packet is found in a protocol item table according to a protocol identifier, determining the found encoding and decoding processor as the corresponding target encoding and decoding processor, wherein the protocol identifier is the identifier of the target protocol of the data packet, and the protocol item table at least comprises the protocol identifier and the encoding and decoding processor corresponding to the protocol identifier; and if the protocol identifier does not find the codec processor corresponding to the data packet in the protocol entry table, determining a preset codec processor as the corresponding target codec processor, so that the processing method of the present application can perform preliminary processing on data by using the preset codec processor even without knowing the communication protocol and the data protocol corresponding to the data packet.
In a specific embodiment of the present application, the protocol entry table may further include a protocol entry identifier, a data index, a data type, and an extension data segment. The data types include integers, floating point numbers, character strings, subprotocol types, function code types and the like. The extension data segment can store corresponding required information in cooperation with the data type, for example, when the data type is a sub-protocol type mark, the protocol identifier of the sub-protocol is stored in the extension data segment.
Specifically, in the actual application process, the product table, the protocol table, and the device table may also be prestored in the server. The product table comprises a product identification column, a communication protocol identification column and a data protocol identification. The protocol table includes a protocol identification column, a protocol type column and a protocol device type column. The device table includes a device identifier column, a product identifier column, and a data protocol identifier column.
In another embodiment of the present application, the processing method further includes: and determining a preset coding and decoding processor as the corresponding target coding and decoding processor under the condition that the target protocol of the data packet is not determined at least according to the module identifier of the target gateway carried by the data packet. In this embodiment, when the communication protocol and the data protocol of the data packet cannot be determined, the preset codec processor is used to determine the target codec processor for the data packet, so that the processing method of the present application can perform preliminary processing on data by using the preset codec processor even without knowing the communication protocol and the data protocol corresponding to the data packet, thereby further ensuring that the processing method of the present application can perform format conversion on the data packet more flexibly.
In order to further obtain the target data more accurately, in another embodiment of the present application, performing format conversion on the data packet at least according to the target codec processor to obtain the target data includes: and performing format conversion on the corresponding data packet according to the target coding and decoding processor and a corresponding data item of the target protocol to obtain the target data, wherein the data item is used for marking the position of a transmission value of the communication protocol or the data protocol.
In a specific embodiment of the present application, for the communication protocol portion of the data packet and the valid data portion of the data protocol, the data items thereof can be described in the following forms: 1. offsetting n bytes, wherein the length is m bytes, the byte order is large end (namely high address)/small end (namely low address), and the byte order is the transmission value of the data item; 2. offsetting n bytes, offsetting the mth bit, wherein the byte order is big end/small end and is the transmission value of the data item; 3. offsetting n bytes, offsetting the mth bit, having a total length of k bits, wherein the byte order is a big end/a small end and is a transmission value of a data item; 4. a data item for which a field name (e.g., JSON) matches, is a transfer value of the data item; 5. the nth element (which can be bit, byte, word, or character string separated by fixed format), the byte order is big end/small end, and is the transmission value of the data item; 6. the nth element (which can be bit, byte, word, or character string separated by fixed format) is m elements long, and the byte order is the transmission value of the data item at the big end/small end; 7. numbering the nth element (which can be bit, byte, word, or character string separated by fixed format), wherein the byte order is big end/small end and is the transmission value of the data item; 8. the nth element (which may be a bit, a byte, a Word, or a string separated in a fixed format) is numbered, the nth element is m long, and the endian is a transmission value of the data item at the big end/the small end. After the actual transmission value is obtained, the coding and decoding processor appointed by the data packet is matched with the relevant parameter of the transmission value in the data item, and the transmission value is converted into the general data type (shaping, floating point number, character string, enumeration character string and the like) which can be directly used in the system.
Because the data packet is regarded as an unlimited level communication protocol and the nesting of the data protocol, the data packet can also be an embedded protocol. In order to further obtain the target data more accurately, in an embodiment of the application, after performing format conversion on the corresponding data packet according to the target codec processor and the corresponding data item of the target protocol to obtain the target data, the processing method further includes: determining whether a sub-protocol identifier exists in the data packet, wherein the sub-protocol identifier is the protocol identifier of a sub-protocol, and the sub-protocol is the communication protocol or the data protocol; determining whether to find the corresponding codec processor in a protocol entry table based on the sub-protocol identifier if the sub-protocol identifier exists in the data packet, where the protocol entry table at least includes the protocol identifier and the codec processor corresponding to the protocol identifier; determining the codec processor as the target codec processor of the corresponding data packet if the corresponding codec processor is found in the protocol entry table based on the sub-protocol identifier, and determining a preset codec processor as the target codec processor of the corresponding data packet if the corresponding codec processor is not found in the protocol entry table based on the sub-protocol identifier; and carrying out format conversion on the data packet based on the target coding and decoding processor to obtain corresponding target data.
In a specific embodiment of the present application, after determining the preset codec processor as the target codec processor and performing format conversion on the corresponding data packet by using the target codec processor, the processing method of the present application may further include: and updating the communication protocol and the data protocol corresponding to the data packet and the corresponding target coding and decoding processor in a protocol entry table so as to facilitate format conversion of the data packet subsequently.
The embodiment of the present application further provides an analysis service, and it should be noted that the analysis service of the embodiment of the present application may be used to execute the processing method for data provided in the embodiment of the present application. The following describes a resolution service provided in an embodiment of the present application.
Fig. 5 is a schematic structural diagram of a resolution service according to an embodiment of the present application. As shown in fig. 5, the resolution service communicates with a plurality of target gateways, one of the target gateways communicates with at least one communication module, and one of the communication modules connects to at least one IoT terminal, and the resolution service includes:
a first determining unit 10, configured to determine a target protocol of a received data packet sent by the target gateway, where the target protocol includes at least one of the following: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
a second determining unit 20, configured to determine a corresponding target codec processor according to at least the target protocol of the data packet, where the target codec processor is configured to perform format conversion on the corresponding data packet;
a first conversion unit 30, configured to perform format conversion on the data packet at least according to the target codec processor, so as to obtain target data.
In the analysis service, after receiving a data packet sent by a target gateway, a first determining unit determines whether the received data packet is an uplink data packet, and if the data packet is the uplink data packet, determines a target protocol of the data packet, that is, determines a communication protocol and a data protocol of the data packet; the second determining unit is used for determining a corresponding target coding and decoding processor at least according to a target protocol of the data packet, namely the communication protocol and/or the data protocol; the first conversion unit is used for performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
Specifically, the data transmission process of the uplink data packet (which may also be referred to as an upload data packet) is as follows: the IoT terminal sends an initial data packet (the initial data packet does not carry a communication protocol at the moment) to the communication module; the communication module performs protocol encapsulation on the received initial data packet based on a communication protocol of the communication module to obtain a preset data packet, and sends the preset data packet to a corresponding target gateway. And when the target gateway receives a preset data packet sent by the communication module, adding a channel unique identifier and a module identifier to the preset data packet to obtain the data packet. The target gateway puts the data packet into a message queue (such as an MQ message queue), and waits until the analysis service consumption is reached, so as to perform format conversion on the data packet to obtain target data. In an actual application process, the analytic service is sent to the target gateway, and then the target gateway sends the analytic service to the communication module, so that a data packet sent to the IoT terminal may be referred to as a downlink data packet (or may also be referred to as an issued data packet).
Specifically, the data packet may also be a data packet obtained after the destination gateway performs packet processing on the data stream or the datagram sent by the communication module. After the target gateway receives the data packet, the target gateway sends the data packet to the resolution service. Under the condition that the target gateway has the identity authentication function, the target gateway can also perform identity authentication on a data packet, so that the data packet subsequently sent to the analysis service can carry a channel unique identifier (namely, a global unique identifier of the Socket). Of course, the unique identifier of the gateway message, the data type of the data packet, the original data encoding format, the communication data (packet/frame), the communication time, and the extension data (e.g., the IP (Internet Protocol Address, IP for short) Address of the source of the data packet, the service level, the will order, etc.) may also be carried.
In a specific embodiment of the present application, the number of target codec processors for performing format conversion on one data packet may be one, or may be multiple. That is, when the format of the packet is converted, the number of target codec processors used may be determined according to the data of the valid data portion in the packet. For example, for a data packet, in the valid data of the data packet, if the first byte carries a temperature value, a target codec processor may be used to perform format conversion on the first byte to obtain a format-converted temperature value; if the second byte to the fifth byte in the valid data carry failure information, another target codec processor can be used to perform format conversion on the second byte to the fifth byte, so as to obtain the failure information after format conversion. Of course, in practical application, the target codec processor may also be adjusted and set according to the data type, coding format, and the like of the valid data in the data packet, and is not limited to the target codec processor mentioned in the above example.
In an actual application process, a communication protocol is added to a received initial data packet of the IoT terminal at the communication module, so as to obtain a preset data packet. After the communication module sends the preset data packet to the target gateway in the form of data stream or datagram, the target gateway needs to perform packet-packet processing on the received data stream or datagram to convert the data stream or datagram into an independent data packet. For a packet, it is essentially a separate packet or frame of data. Which may carry a data packet header, a data packet payload, and a data packet checksum or data packet trailer, among other things. In the processing method, the protocol used by the data packet is divided into a communication protocol and a data protocol, so that the data packet received by the target gateway is the communication protocol of unlimited level and the nesting of the data protocol. Specifically, as shown in fig. 2, for example, a data packet 100 includes a communication protocol header 101 and a communication protocol payload 104, and the communication protocol payload 104 includes an embedded data protocol portion 102 and a check tail 103. As shown in fig. 3, the communication protocol payload 104 may further include a first data protocol 105, a first data payload 111, and a transmission check tail 110 (in an actual application process, the communication protocol payload 104 may also not include the transmission check tail 110), and for the first data payload 111, the first transmission data 106 and the second transmission data 107 may be embedded inside, and the sub-protocol 108 and the sub-protocol data 109 (i.e., the valid data portion of the sub-protocol) may be embedded inside. As shown in fig. 4, the communication protocol payload 104 may further include a second data protocol 112 and a second data payload 113, wherein the second data payload 113 may further embed the plurality of transmission data.
In an embodiment of the application, the data packet carries a channel unique identifier and a module identifier, the channel unique identifier is a global unique identifier of a connection established between the communication module and the target gateway, the module identifier is a unique identifier of the target gateway, and the first determining unit includes a first determining module and a second determining module, where the first determining module is configured to determine the target protocol based on the channel unique identifier of the data packet when the corresponding target protocol can be determined based on the channel unique identifier of the data packet; the second determining module is configured to determine the target protocol of the data packet according to at least the module identifier carried by the data packet under the condition that the corresponding target protocol cannot be determined based on the channel unique identifier of the data packet, which ensures that an analysis service can determine a communication protocol and a data protocol of the data packet relatively easily, and ensures that the calculation amount of the analysis service is small.
In order to determine a communication protocol and a data protocol used by a data packet more simply, in another embodiment of the present application, the second determining module includes a first determining submodule, a searching submodule, and a second determining submodule, where the first determining submodule is configured to determine the communication protocol according to the module identifier of the data packet; the searching submodule is used for searching a corresponding product identifier in a communication module table according to the module identifier of the data packet, wherein the communication module table at least comprises the module identifier and a product identifier, and the product identifier is an identifier of a product formed by the communication protocol and the data protocol; the second determining submodule is configured to determine the data protocol of the corresponding data packet according to the communication protocol and the product identifier. That is to say, according to the module identifier of the target gateway, the outer layer communication protocol of the data packet can be obtained, then the product identifier corresponding to the communication protocol is obtained based on the pre-stored communication module table, the data protocol corresponding to the data packet is obtained, and further the data protocol and the communication protocol corresponding to the data packet are determined relatively simply and accurately.
Specifically, in the actual application process, the communication module table is preset in the parsing service. The communication module table has a module identification column, a product identification column and a communication protocol identification column. Wherein the product identification column has a plurality of products therein. The products are composed of different communication protocols and data protocols. In a specific embodiment of the present application, communication protocol a and data protocol a constitute one product, communication protocol a and data protocol B constitute another product, and so on.
In order to determine a target codec processor corresponding to a data packet more simply, in another embodiment of the present application, the second determining unit includes a third determining module and a fourth determining module, where the third determining module is configured to determine, when the codec processor corresponding to the data packet is found in a protocol entry table according to a protocol identifier, the found codec processor as the corresponding target codec processor, where the protocol identifier is an identifier of the communication protocol and/or the data protocol of the data packet, and the protocol entry table at least includes the protocol identifier and the codec processor corresponding to the protocol identifier; the fourth determining module is configured to determine a preset codec processor as the corresponding target codec processor if the codec processor corresponding to the data packet is not found in the protocol entry table according to the protocol identifier, so that the processing method of the present application can perform preliminary processing on data by using the preset codec processor even without knowing a communication protocol and a data protocol corresponding to the data packet.
In a specific embodiment of the present application, the protocol entry table may further include a protocol entry identifier, a data index, a data type, and an extension data segment. The data types include integers, floating point numbers, character strings, subprotocol types and function code types. The extension data segment can store corresponding required information in cooperation with the data type, for example, when the data type is a sub-protocol type mark, the protocol identifier of the sub-protocol is stored in the extension data segment.
In yet another embodiment of the application, the parsing service further includes a third determining unit, configured to determine a preset codec processor as the corresponding target codec processor when the target protocol of the data packet is not determined at least according to the module identifier of the target gateway carried by the data packet. In this embodiment, when the communication protocol and the data protocol of the data packet cannot be determined, the preset codec processor is used to determine the target codec processor for the data packet, so that the processing method of the present application can perform preliminary processing on data by using the preset codec processor even without knowing the communication protocol and the data protocol corresponding to the data packet, thereby further ensuring that the processing method of the present application can perform format conversion on the data packet more flexibly.
In order to further obtain the target data more accurately, in another embodiment of the present application, the first converting unit includes a converting module, configured to perform format conversion on the corresponding data packet according to the target codec processor and a corresponding data item of the target protocol to obtain the target data, where the data item is used to mark a position of a transmission value of the communication protocol or the data protocol.
In a specific embodiment of the present application, for the communication protocol portion of the data packet and the valid data portion of the data protocol, the data items thereof can be described in the following forms: 1. offsetting n bytes, wherein the length is m bytes, the byte order is large end (namely high address)/small end (namely low address), and the byte order is the transmission value of the data item; 2. offsetting n bytes, offsetting the mth bit, wherein the byte order is big end/small end and is the transmission value of the data item; 3. offsetting n bytes, offsetting the mth bit, having a total length of k bits, wherein the byte order is a big end/a small end and is a transmission value of a data item; 4. a data item for which a field name (e.g., JSON) matches, is a transfer value of the data item; 5. the nth element (which can be bit, byte, word, or character string separated by fixed format), the byte order is big end/small end, and is the transmission value of the data item; 6. the nth element (which can be bit, byte, word, or character string separated by fixed format) is m long, and the byte order is the transmission value of the data item at the big end/small end; 7. numbering the nth element (which can be bit, byte, word, or character string separated by a fixed format), wherein the byte order is a big end/a small end and is a transmission value of a data item; 8. the nth element (which may be a bit, a byte, a Word, or a string separated in a fixed format) is numbered, the nth element is m long, and the endian is a transmission value of the data item at the big end/the small end. After the actual transmission value is obtained, the coding and decoding processor appointed by the data packet is matched with the relevant parameter of the transmission value in the data item, and the transmission value is converted into the general data type (shaping, floating point number, character string, enumeration character string and the like) which can be directly used in the system.
Because the data packet is regarded as an unlimited level communication protocol and the nesting of the data protocol, the data packet can also be an embedded protocol. In order to further obtain target data more accurately, in an embodiment of the present application, after format conversion is performed on the corresponding data packet according to the target codec processor and the corresponding data item of the communication protocol and/or the data protocol to obtain the target data, the parsing service further includes a fourth determining unit, a fifth determining unit, a searching unit, and a second converting unit, where the fourth determining unit is configured to determine whether a sub-protocol identifier exists in the data packet, where the sub-protocol identifier is the protocol identifier of a sub-protocol, and the sub-protocol is the communication protocol or the data protocol; the fifth determining unit is configured to determine whether to find the corresponding codec processor in a protocol entry table based on the sub-protocol identifier when the sub-protocol identifier exists in the data packet, where the protocol entry table at least includes the protocol identifier and the codec processor corresponding to the protocol identifier; the searching unit is configured to determine the codec processor as the target codec processor of the corresponding data packet if the corresponding codec processor is searched in the protocol entry table based on the sub-protocol identifier, and determine a preset codec processor as the target codec processor of the corresponding data packet if the corresponding codec processor is not searched in the protocol entry table based on the sub-protocol identifier; the second conversion unit is configured to perform format conversion on the data packet based on the target codec processor to obtain the corresponding target data.
In a specific embodiment of the present application, after determining the preset codec processor as the target codec processor and performing format conversion on the corresponding data packet by using the target codec processor, the processing method of the present application may further include: and updating the communication protocol and the data protocol corresponding to the data packet and the corresponding target coding and decoding processor in a protocol entry table so as to facilitate format conversion of the data packet subsequently.
The analysis service comprises a processor and a memory, the first determining unit, the second determining unit, the first converting unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the problem of poor flexibility of data processing in the prior art is solved by adjusting kernel parameters.
The memory may include volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), including at least one memory chip.
An embodiment of the present invention provides a computer-readable storage medium, on which a program is stored, where the program, when executed by a processor, implements a processing method of the above data.
The embodiment of the invention provides a processor, which is used for running a program, wherein the data processing method is executed when the program runs.
In a typical embodiment of the application, an internet of things platform is also provided. As shown in fig. 6, the internet of things platform includes an access service 116, a resolution service 117, and a business service 118. The access service 116 includes a plurality of target gateways 119, and the target gateways 119 are configured to send the received data packets sent by the IoT terminal 114 to the parsing service 117; the analysis service 117 communicates with the access service 116, and the analysis service 117 is configured to execute any one of the data processing methods; the business service 118 communicates with the parsing service 117, and is configured to obtain the target data converted by the parsing service 117.
The internet of things platform comprises an access service, an analysis service and a business service, wherein the analysis service is used for executing any one of the data processing methods. In the data processing method, firstly, after receiving a data packet sent by a target gateway, determining whether the received data packet is an uplink data packet, and if the data packet is the uplink data packet, determining a target protocol of the data packet, namely determining a communication protocol and a data protocol of the data packet; then, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, namely the communication protocol and/or the data protocol; and finally, performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by a service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
In a specific embodiment of the present application, as shown in fig. 6, the access service 116 includes a plurality of target gateways 119 for providing connection support for communication modules 115 with different communication protocols. E.g. different TCP's private protocol access service, MQTT access service, coAP access service, HTTP pull service (an implementation of third party data access), etc. Meanwhile, the access service 116 and the parsing service 117 communicate through a message queue, that is, the target gateway 119 puts the received data packet into the message queue, so that the parsing service 117 consumes the data packet from the message queue and performs format conversion. If the target gateway 119 itself has the capability of authenticating the identity of the IoT terminal, the target gateway 119 performs authentication to implement the identity at this time, and sends the authentication result to the message queue.
In an actual application process, as shown in fig. 6, one communication module 115 may communicate with multiple IoT terminals 114, but of course, one communication module 115 may also communicate with only one IoT terminal 114.
Specifically, as shown in fig. 6, the number of IoT terminals 114 connected to one communication module 115 is not limited in the present application. Specifically, the IoT terminal may have a communication capability, for example, the IoT terminal may support a communication module of a 4G/5G/NBIoT network. Of course, the IoT terminal may not have communication capability. Specifically, the communication method between the business service 118 and the parsing service 117 is not limited to remote invocation, but may be to transmit a message through a message queue to improve the adaptability to the burst traffic.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized:
step S101, determining a target protocol of the data packet when the received data packet sent by the target gateway is an uplink data packet, where the target protocol includes at least one of the following: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
step S102, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet;
step S103, at least according to the target coding and decoding processor, carrying out format conversion on the data packet to obtain target data.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program initialized with at least the following method steps when executed on a data processing device:
step S101, determining a target protocol of the data packet when the received data packet sent by the target gateway is an uplink data packet, where the target protocol includes at least one of the following: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
step S102, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet;
step S103, at least according to the target coding and decoding processor, carrying out format conversion on the data packet to obtain target data.
In order to make the technical solutions of the present application more clearly understood by those skilled in the art, the technical solutions and technical effects of the present application will be described below with reference to specific embodiments.
Examples
The present embodiment relates to a data processing scheme, as shown in fig. 7. The scheme is applied in a resolution service (a kind of server). First, a data packet sent by a target gateway is received. And after receiving the data packet sent by the target gateway, determining whether the received data packet is an uplink data packet. And entering other message type processing flows under the condition that the received data packet is not an uplink data packet. If the received data packet is an uplink data packet, the process of converting the format of the data packet is performed. The method comprises the following specific steps:
the first step is as follows: determining a communication protocol and a data protocol used by a corresponding data packet based on the channel unique identifier carried by the data packet under the condition that the communication protocol and the data protocol of the corresponding data packet can be determined based on the channel unique identifier carried by the data packet; and under the condition that the communication protocol and the data protocol of the corresponding data packet cannot be determined based on the channel unique identifier carried by the data packet, determining the communication protocol and the data protocol used by the data packet at least according to the module identifier (namely the source of the data packet) of the target gateway carried by the data packet.
The second step: after determining a communication protocol and a data protocol used by a data packet at least according to a module identifier of a target gateway (namely, a source of the data packet) carried by the data packet, or determining the communication protocol and the data protocol used by a corresponding data packet based on a channel unique identifier carried by the data packet, determining an encoding and decoding processor corresponding to the data packet based on the communication protocol and/or the data protocol of the data packet, and determining the determined encoding and decoding processor as a target encoding and decoding processor; and under the condition that the communication protocol and the data protocol used by the data packet cannot be determined at least according to the module identifier of the target gateway (namely the source of the data packet) carried by the data packet, determining the preset coding and decoding processor as the target coding and decoding processor of the corresponding data packet.
The third step: and according to the target coding and decoding processor of the data packet and the corresponding data item, carrying out format conversion on the data packet to obtain target data.
The fourth step: and after format conversion is carried out on the data packet according to the target coding and decoding processor of the data packet and the corresponding data item to obtain target data, whether the sub-protocol identification exists in the corresponding data packet is determined.
The fifth step: determining whether to find out a corresponding encoding and decoding processor in a protocol item table based on the subprotocol identifier under the condition that the subprotocol identifier exists in the corresponding data packet; determining an encoding and decoding processor as the target encoding and decoding processor of the corresponding data packet under the condition that the corresponding encoding and decoding processor is found in the protocol item table based on the sub-protocol identifier, and determining a preset encoding and decoding processor as the target encoding and decoding processor of the corresponding data packet under the condition that the corresponding encoding and decoding processor is not found in the protocol item table based on the sub-protocol identifier; and carrying out format conversion on the data packet based on the target coding and decoding processor to obtain corresponding target data.
In the above embodiments of the present invention, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described in detail in a certain embodiment.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be an indirect coupling or communication connection through some interfaces, units or modules, and may be electrical or in other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.
From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:
1) In the data processing method of the present application, first, after receiving a data packet sent by a target gateway, it is determined whether the received data packet is an uplink data packet, and if the data packet is the uplink data packet, a target protocol of the data packet, that is, a communication protocol and a data protocol of the data packet, is determined; then, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, namely the communication protocol and/or the data protocol; and finally, performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by a service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more easily. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
2) In the analysis service of the application, a first determining unit determines whether a received data packet is an uplink data packet after receiving the data packet sent by a target gateway, and determines a target protocol of the data packet, namely determines a communication protocol and a data protocol of the data packet when the data packet is the uplink data packet; the second determining unit is used for determining a corresponding target coding and decoding processor at least according to a target protocol of the data packet, namely the communication protocol and/or the data protocol; the first conversion unit is used for performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
3) The Internet of things platform comprises an access service, an analysis service and a business service, wherein the analysis service is used for executing any one of the data processing methods. In the data processing method, firstly, after receiving a data packet sent by a target gateway, determining whether the received data packet is an uplink data packet, and if the data packet is the uplink data packet, determining a target protocol of the data packet, namely determining a communication protocol and a data protocol of the data packet; then, determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, namely the communication protocol and/or the data protocol; and finally, performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data, namely converting the data packet into a data format which can be read by a service. According to the data processing method, the protocol used by the data packet is regarded as any nesting of the communication protocol and the data protocol, the communication protocol and the data protocol of the data packet are determined under the condition that the received data packet is an uplink data packet, the corresponding target coding and decoding processor is determined according to the communication protocol and/or the data protocol of the data packet, and format conversion is performed on the corresponding data packet at least according to the target coding and decoding processor to obtain target data, so that format conversion is performed on effective data carried by the data packet more simply. That is to say, the processing method of the present application can determine the corresponding target codec processor to implement format conversion on the data packet under the condition of knowing the communication protocol and the data protocol corresponding to the data packet, so that the processing method of the present application can process any type of data packet, implement more flexible and more reliable processing on data, and solve the problem of poor flexibility of data processing in the prior art.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. A method for processing data, the method being applied in a resolution service, the resolution service communicating with a plurality of target gateways, one of the target gateways communicating with at least one communication module, one of the communication modules connecting with at least one IoT terminal, the method comprising:
determining a target protocol of the data packet under the condition that the received data packet sent by the target gateway is an uplink data packet, wherein the target protocol comprises at least one of the following protocols: a communication protocol, a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
determining a corresponding target coding and decoding processor at least according to the target protocol of the data packet, wherein the target coding and decoding processor is used for carrying out format conversion on the corresponding data packet;
and performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data.
2. The processing method according to claim 1, wherein the data packet carries a channel unique identifier and a module identifier, the channel unique identifier is a global unique identifier of a connection established between the communication module and the target gateway, the module identifier is a unique identifier of the target gateway, and when the received data packet sent by the target gateway is an uplink data packet, determining a target protocol of the data packet includes:
determining the target protocol based on the channel unique identifier of the data packet under the condition that the corresponding target protocol can be determined based on the channel unique identifier of the data packet;
and under the condition that the corresponding target protocol cannot be determined based on the channel unique identifier of the data packet, determining the target protocol of the data packet at least according to the module identifier carried by the data packet.
3. The processing method according to claim 2, wherein determining the target protocol of the packet according to at least the module identifier carried by the packet comprises:
determining the communication protocol according to the module identification of the data packet;
searching a corresponding product identifier in a communication module table according to the module identifier of the data packet, wherein the communication module table at least comprises the module identifier and the product identifier, and the product identifier is an identifier of a product formed by the communication protocol and the data protocol;
and determining the data protocol of the corresponding data packet according to the communication protocol and the product identification.
4. The processing method of claim 1, wherein determining a corresponding target codec processor based at least on the target protocol of the data packet comprises:
under the condition that an encoding and decoding processor corresponding to the data packet is found in a protocol item table according to a protocol identifier, determining the found encoding and decoding processor as the corresponding target encoding and decoding processor, wherein the protocol identifier is the identifier of the target protocol of the data packet, and the protocol item table at least comprises the protocol identifier and the encoding and decoding processor corresponding to the protocol identifier;
and under the condition that the coding and decoding processor corresponding to the data packet is not found in the protocol item table according to the protocol identifier, determining a preset coding and decoding processor as the corresponding target coding and decoding processor.
5. The processing method according to claim 2, characterized in that it further comprises:
and determining a preset coding and decoding processor as the corresponding target coding and decoding processor under the condition that the target protocol of the data packet is not determined at least according to the module identification of the target gateway carried by the data packet.
6. The processing method according to any one of claims 1 to 5, wherein performing format conversion on the data packet according to at least the target codec processor to obtain target data comprises:
and performing format conversion on the corresponding data packet according to the target coding and decoding processor and the corresponding data item of the target protocol to obtain the target data, wherein the data item is used for marking the position of the transmission value of the communication protocol or the data protocol.
7. The processing method according to claim 6, wherein after performing format conversion on the corresponding data packet according to the target codec processor and the corresponding data item of the target protocol to obtain the target data, the processing method further comprises:
determining whether a sub-protocol identifier exists in the data packet, wherein the sub-protocol identifier is the protocol identifier of a sub-protocol, and the sub-protocol is the communication protocol or the data protocol;
determining whether to find out the corresponding coding and decoding processor in a protocol item table based on the sub-protocol identifier under the condition that the sub-protocol identifier exists in the data packet, wherein the protocol item table at least comprises the protocol identifier and the coding and decoding processor corresponding to the protocol identifier;
determining the coding and decoding processor as the target coding and decoding processor of the corresponding data packet under the condition that the corresponding coding and decoding processor is found in the protocol item table based on the sub-protocol identification, and determining a preset coding and decoding processor as the target coding and decoding processor of the corresponding data packet under the condition that the corresponding coding and decoding processor is not found in the protocol item table based on the sub-protocol identification;
and carrying out format conversion on the data packet based on the target coding and decoding processor to obtain the corresponding target data.
8. A resolution service, the resolution service in communication with a plurality of target gateways, one of the target gateways in communication with at least one communication module, one of the communication modules in communication with at least one IoT terminal, the resolution service comprising:
a first determining unit, configured to determine a target protocol of a received data packet sent by the target gateway when the received data packet is an uplink data packet, where the target protocol includes at least one of: a communication protocol and a data protocol, wherein the data protocol is a protocol describing data of the IoT terminal;
a second determining unit, configured to determine a corresponding target codec processor according to at least the target protocol of the data packet, where the target codec processor is configured to perform format conversion on the corresponding data packet;
and the first conversion unit is used for performing format conversion on the data packet at least according to the target coding and decoding processor to obtain target data.
9. A computer-readable storage medium characterized by comprising a stored program, wherein the program executes a method of processing data according to any one of claims 1 to 7.
10. An internet of things platform, comprising:
the access service comprises a plurality of target gateways, and the target gateways are used for sending the received data packets sent by the IoT terminals to the analysis service;
the resolution service, which is in communication with the access service, and is used for executing the data processing method of any one of claims 1 to 7;
and the business service is communicated with the analysis service and is used for acquiring the target data converted by the analysis service.
CN202211304440.7A 2022-10-24 2022-10-24 Data processing method, analysis service and Internet of things platform Pending CN115665287A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211304440.7A CN115665287A (en) 2022-10-24 2022-10-24 Data processing method, analysis service and Internet of things platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211304440.7A CN115665287A (en) 2022-10-24 2022-10-24 Data processing method, analysis service and Internet of things platform

Publications (1)

Publication Number Publication Date
CN115665287A true CN115665287A (en) 2023-01-31

Family

ID=84992375

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211304440.7A Pending CN115665287A (en) 2022-10-24 2022-10-24 Data processing method, analysis service and Internet of things platform

Country Status (1)

Country Link
CN (1) CN115665287A (en)

Similar Documents

Publication Publication Date Title
US10313410B2 (en) Systems and methods using binary dynamic rest messages
CN109525661B (en) Data transmission method and device
US9560170B2 (en) System and method of abstracting communication protocol using self-describing messages
CA2578856C (en) System and method for transmitting acars messages over a tcp/ip data communication link
US7333801B2 (en) Method and apparatus for translating resource names in a wireless environment
US9462085B2 (en) Chunk-based communication of binary dynamic rest messages
CA2468667A1 (en) System and method for identifying and accessing network services
CN111083161A (en) Data transmission processing method and device and Internet of things equipment
WO2017000915A1 (en) Multicast transmission method, information extraction method and corresponding terminal and device
CN109688164A (en) Method for converting protocol and device
CN105376225A (en) Software-as-a-service method and device
CN104486396A (en) Cross-platform network data transmission method
CN114338827A (en) Internet of things protocol adaptation gateway, method and readable storage medium
CN103532959A (en) Cross-platform network communication method
CN108900433B (en) Method, terminal and server for transmitting lora message
US20100303074A1 (en) Apparatus and method for supporting higher data rates on links with variable frame sizes
US20070211699A1 (en) Wireless services provider network system and method
CN103647763A (en) Mobile terminal advertisement invoking method and system
DE50210901D1 (en) BROWSIBLE COMMUNICATION SYSTEM AND CLIENT AND SERVER FOR SUCH A COMMUNICATION SYSTEM
CN115665287A (en) Data processing method, analysis service and Internet of things platform
CN109005250B (en) Method and device for accessing server
CN109298866A (en) TLV format protocol fast resolving method based on C language
CN110071925B (en) Data transmission method and corresponding server and computer readable storage medium
CN107079423A (en) A kind of data transmission method and equipment
CN108259500A (en) A kind of RPC means of communication, system and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination