CN112765175A - Interface data processing method and device, computer equipment and medium - Google Patents

Interface data processing method and device, computer equipment and medium Download PDF

Info

Publication number
CN112765175A
CN112765175A CN202110081553.4A CN202110081553A CN112765175A CN 112765175 A CN112765175 A CN 112765175A CN 202110081553 A CN202110081553 A CN 202110081553A CN 112765175 A CN112765175 A CN 112765175A
Authority
CN
China
Prior art keywords
interface
data
field
context information
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110081553.4A
Other languages
Chinese (zh)
Other versions
CN112765175B (en
Inventor
谢靖
孟鑫
田野
梁彧
傅强
王杰
杨满智
蔡琳
金红
陈晓光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Eversec Beijing Technology Co Ltd
Original Assignee
Eversec Beijing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Eversec Beijing Technology Co Ltd filed Critical Eversec Beijing Technology Co Ltd
Priority to CN202110081553.4A priority Critical patent/CN112765175B/en
Publication of CN112765175A publication Critical patent/CN112765175A/en
Application granted granted Critical
Publication of CN112765175B publication Critical patent/CN112765175B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a method and a device for processing interface data, computer equipment and a medium. The method comprises the steps of receiving interface data, and determining the interface type of the interface data according to an interface type field in the interface data; inquiring the global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type; the interface data are analyzed according to the field table in the target data interface table, the analysis result is written into the context information corresponding to the target data interface table in the global data interface table, the uniform interface data analysis function is realized, the problems that the development, debugging, testing and the like are required to be invested when a data interface is added due to the fact that the interface data analysis operation is carried out independently are solved, and the development period, the debugging cost and the later maintenance cost are greatly reduced.

Description

Interface data processing method and device, computer equipment and medium
Technical Field
The present invention relates to data processing technologies, and in particular, to a method and an apparatus for processing interface data, a computer device, and a medium.
Background
The collection server may collect data traffic from the plurality of data interfaces by using a DPI (deep packet inspection) technology, and transmit the collected interface data to the composition server. The synthesis server identifies the interface protocol of the received interface data, then carries out data analysis and data extraction operation on the received interface data according to the interface data specifications corresponding to different types of interface data, and stores the extracted data in the directory of the corresponding data interface.
Because the analysis operation is realized by adopting respective interface data specifications for different types of data interfaces, the development period, the debugging cost and the maintenance cost are influenced because manpower development, debugging, testing and the like are required to be invested every time a data interface is added.
Disclosure of Invention
The embodiment of the invention provides a method and a device for processing interface data, computer equipment and a medium, which can optimize the analysis mode of the interface data in the related technology.
In a first aspect, an embodiment of the present invention provides a method for processing interface data, including:
receiving interface data, and determining the interface type of the interface data according to an interface type field in the interface data;
inquiring a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, wherein the global data interface table is generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces;
and analyzing the interface data according to a field table in the target data interface table, and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
In a second aspect, an embodiment of the present invention further provides an apparatus for processing interface data, where the apparatus includes:
the type determining module is used for receiving interface data and determining the interface type of the interface data according to an interface type field in the interface data;
the interface table query module is used for querying a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, wherein the global data interface table is generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces;
and the data analysis module is used for analyzing the interface data according to the field table in the target data interface table and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a memory for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing interface data according to any embodiment of the present invention.
In a fourth aspect, the embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are used for executing the processing method of the interface data according to any embodiment of the present invention.
The embodiment of the invention provides a method, a device, computer equipment and a medium for processing interface data, wherein the interface type is determined through an interface type field contained in the interface data, a pre-configured global data interface table is inquired based on the interface type to obtain a corresponding target data interface table, the interface data is analyzed according to the field table in the target data interface table, and an analysis result is written into context information corresponding to the target data interface table in the global data interface table, so that a uniform interface data analysis function is realized, the problems that manpower development, debugging, testing and the like are required to be invested when a data interface is newly added due to the fact that interface data analysis operation is carried out independently are solved, and the development period, the debugging cost and the later maintenance cost are greatly reduced.
Drawings
Fig. 1 is a flowchart of a method for processing interface data according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a generation flow of a global data interface table according to an embodiment of the present invention;
fig. 3 is a flowchart of another interface data processing method according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a backfill process of interface data according to an embodiment of the present invention;
fig. 5 is a block diagram of an interface data processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
For ease of understanding, the related concepts of interface data parsing and backfilling will first be described.
The types of interface data collected by the DPI and received by the composition server may be many, and each interface data needs to be analyzed and data extraction is performed. For example, the signaling plane data includes N1, N2, N4, N7, N8, N10, N11, N12, N14, N15, N22, N26, etc.; the user plane data comprises n3http, n3https, n3mail, n3dns, n3ftp, n3sip, n3rtsp, n3im, n3video, n3mms, n3 coach, n3general and the like; the wireless interfaces include E1, F1, X2, UU, XN, UEMR, CELLMR, etc.
In practical application, some field data in some data interfaces are not carried by themselves, and need to be acquired from other data interfaces, and then are backfilled to corresponding field data of the data interfaces. For example, signaling interface data backfill: the N10 interface data do not carry TAC and CELL ID, and need to be acquired from the N1N2 interface data. User interface data backfilling: the data of the N3 interface does not carry IMSI, PEI, MSISDN and RAT, and needs to be obtained from the data of the N1N2, N4 and N11 interfaces. Wireless interface data backfilling: the wireless interface data does not carry information such as mobile phone numbers, IMSIs, PEI and the like, and needs to be acquired from the signaling interface.
The existing interface data analysis technology generally analyzes the corresponding type of interface data individually according to respective interface specifications. In the existing interface data backfilling technology, a certain space is generally reserved for a field to be backfilled in the interface data, so that the data is backfilled to a corresponding position.
For the existing interface data analysis technology, when a data interface is added, manpower development, debugging, testing and the like are required to be input, and the stability of the system is not facilitated. This results in a long development cycle, a very large amount of time and labor cost, and a high post-maintenance cost.
Aiming at the existing interface data backfilling technology, a certain space is reserved for the field data to be backfilled, so that the field data which does not carry useful information exists in the interface data, and the field data occupies network bandwidth when being transmitted in a network, thereby greatly reducing the utilization rate of the network bandwidth.
Fig. 1 is a flowchart of an interface data processing method according to an embodiment of the present invention, where the embodiment is applicable to a mobile network data synthesis case, and the method may be executed by an interface data processing apparatus, which may be implemented by software and/or hardware and is generally configured in a computer device such as a synthesis server. As shown in fig. 1, the method includes:
step 110, receiving interface data, and determining an interface type of the interface data according to an interface type field in the interface data.
The interface data is data traffic transmitted through the data interface. Illustratively, the interface data includes signaling plane data, user plane data, radio interface data, and the like. The collection server collects data traffic, i.e. interface data, from each data interface by a DPI probe technique. The acquisition server then transmits the interface data to the composition server.
The interface type field is used to indicate the interface type of the corresponding interface data. For example, an interface type field is contained in a common header of the interface data. The interface type of the interface data may be determined by an interface type field in a common header of the interface data.
The interface type is used to indicate the type of data interface. For example, the interface types may include a signaling plane interface, a user plane interface, a wireless interface, and the like. The signaling interface includes N1, N2, N4, N7, N8, N10, N11, N12, N14, N15, N22, N26, and the like. The user interface comprises n3http, n3https, n3mail, n3dns, n3ftp, n3sip, n3rtsp, n3im, n3video, n3mms, n3 coach, n3general and the like. The wireless interface includes E1, F1, X2, UU, XN, UEMR, CELLMR, etc.
Assuming that the interface field in the common header of the interface data is an interface type field for indicating the interface type of the interface data, and if the interface is N10, it is determined that the interface type is an N10 type signaling plane interface.
Specifically, interface data is received, and an interface type field in the interface data is analyzed to obtain the content of the interface type field. And determining the interface type of the data interface corresponding to the interface data according to the interface type field content.
And 120, inquiring a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type.
The global data interface table is a data table generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces. The field format in all mobile network data interfaces can be described by adopting a universal configuration format through the global data table. For example, the global data interface table includes a data interface table, a field table, and a context information table; the interface data table is used for describing the structural information of the data interface table; the field table is used for describing configuration information of fields of the data interface; the context information table is used to describe context information of fields of the data interface and/or context information of sub-tables of fields of the data interface.
Optionally, the data interface table includes information such as a name of the data interface (or referred to as a data interface type), the number of fields of the data interface, and a description of each field. The field table comprises field name, type, length, tag value, decoding type, sub-table mark, sub-table name, sub-table id and other information. The context information table includes a context information table of fields of the data interface and a sub-table context information table of fields of the data interface. The context information table of the fields of the data interface includes information such as an omission identifier (or called an index), a data start address, a data length, a data offset, the number of groups (or called sub-tables), the number of fields of each group (or called a sub-table), a sub-table id, and a sub-table array structure. The sub-table context information table of the fields of the data interface includes information such as an omission identifier (or called an index), a data start address, a data length, and a data offset.
Illustratively, taking a 5G data interface as an example, a data interface configuration file of each data interface is generated according to the 5G data interface specification, and fields in the data interface are stored in a common configuration format in the data interface configuration file. And loading all the data interface configuration files through a compiling tool, generating a global data interface table, and loading the pre-generated global data interface table to the memory when the initialization of the synthesis program is detected. Fig. 2 is a schematic diagram of a generation flow of a global data interface table according to an embodiment of the present invention. As shown in fig. 2, the corresponding data interface configuration file is generated by copying and pasting the entire data interface specification of the data interface, that is, the data interface configuration file is generated based on the data interface specification doc. And compiling all the data interface configuration files through a compiling tool to obtain a global data interface table, namely compiling the data interface configuration files csv n to obtain the global data interface table 5 g.db. And when the synthesis server detects the initialization of the synthesis program, loading the pre-generated global data interface table 5g.db into the memory.
The target data interface table is a data structure required for analyzing the received interface data. For example, a data interface table having a data interface name identical to the contents of the interface type field in the received interface data is used as the target data interface table.
Exemplarily, inquiring each interface type identifier in the global data interface table according to the interface type to obtain an interface type identifier matched with the interface type; and determining a target data interface table corresponding to the interface type according to the corresponding relation between the interface type identifier in the global data table and the interface data table. The specific query of each interface type identifier in the global data interface table according to the interface type may be: and inquiring the interface name contained in each data interface table in the global data interface table according to the content of the interface type field. And the interface type identifier matched with the interface type can be an interface name consistent with the content of the interface type field in the data interface table. As mentioned above, the interface data table contains interface names, so that the target data interface table corresponding to the interface type can be determined according to the interface name consistent with the content of the interface type field.
Step 130, analyzing the interface data according to the field table in the target data interface table, and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
Wherein the field table is a data structure describing configuration information of fields of the data interface. It should be noted that the global data interface table includes a plurality of data interface tables, each data interface table includes a field table, and information included in the field table is described in the above steps, which is not described herein again.
The analysis result is result information obtained by analyzing the received interface data according to the configuration information in the field table. For interface data that does not contain a data group (i.e., the number of elements in the group is greater than 1), the parsing result may include information such as an omission flag, a data start address, a data length, and a data offset. For the interface data containing the sub-table, the analysis result may include information such as an omission identifier, a data start address, a data length, a data offset, a number of groups, a number of fields in each group, a sub-table id, and a sub-table array structure. The analysis result of the sub-table includes the omission flag, the data start address, the data length, the data offset and other information of the sub-table.
The global data interface table comprises a data interface table and a context information table, and the data interface table and the context information table have an association relation. The association is a context information table for describing context information of fields of the data interface and/or context information of sub-tables of fields of the data interface. After the interface data is analyzed, the analysis result can be written into the context information corresponding to the target data interface table. For example, the parsing result may be written into a context information table corresponding to the target data interface table.
Illustratively, interface data is analyzed according to configuration information of fields included in a field table in the target data interface table to obtain context information; and writing the context information into a context information table corresponding to the target data interface table in the global data interface table. Optionally, when the interface data includes at least one group of data whose element number is greater than 1, analyzing the interface data according to configuration information of fields included in a field table in the target data interface table to obtain context information of the fields and context information of each sub-table; writing the context information of the field into a field context information table corresponding to a target data interface table in the global data interface table; and writing the context information of each sub-table into a sub-table context information table corresponding to the target data interface table in the global interface table.
For example, after receiving interface data of type N22, obtaining that the field content of the interface field in the common header is N22, and querying the global data interface table based on N22 to obtain a target data interface table corresponding to the interface data of type N22. Because the data interface table comprises a field table, the interface data is analyzed based on the configuration information of fields such as the field name, the type, the length, the tag value, the decoding type, the sub-table mark, the sub-table name, the sub-table id and the like in the field table, and the analysis result of the interface data, namely the context information of each field of the interface data, is obtained. And writing the context information of the field into a context information table corresponding to the target data interface table. Optionally, for the interface data containing the sub-table, the parsing result further includes context information of the sub-table. And respectively writing the context information of the fields and the context information of the sub-tables into the context information table and the sub-table context information table of the target data interface table.
According to the technical scheme of the embodiment, the interface type is determined through the interface type field contained in the interface data, the pre-configured global data interface table is inquired based on the interface type to obtain the corresponding target data interface table, the interface data is analyzed according to the field table in the target data interface table, the analysis result is written into the context information corresponding to the target data interface table in the global data interface table, the uniform interface data analysis function is achieved, the problems that every new data interface is added due to independent interface data analysis operation, manpower development, debugging, testing and the like are required to be invested are avoided, and the development period, the debugging cost and the later maintenance cost are greatly reduced.
Fig. 3 is a flowchart of another interface data processing method according to an embodiment of the present invention, in which a backfill step of omitting fields of the interface data is added based on the above embodiment. As shown in fig. 3, the method includes:
step 310, receiving interface data, and determining an interface type of the interface data according to an interface type field in the interface data.
And 320, inquiring a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type.
Step 330, analyzing the interface data according to the field table in the target data interface table, and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
Step 340, after receiving the interface data, decomposing the interface data according to the data transmission format, and caching the obtained interface data segments in the pre-allocated memory respectively.
Wherein, the data transmission format is used for indicating the organization form or the encapsulation form of the interface data. For example, assuming that the acquisition server transmits data in the form of "in SDTP protocol header + pieces of XDR data", the data transmission format may be "in SDTP protocol header + pieces of XDR data".
The interface data segment may be understood as a decomposition result obtained by decomposing the interface data according to the data transmission format. It should be noted that each interface data segment can be regarded as complete XDR data. For example, for interface data transmitted in the form of SDTP protocol header + multiple pieces of XDR data, the interface data is decomposed based on the data transmission format to obtain multiple pieces of XDR data. Fig. 4 is a schematic diagram of a backfill process of interface data according to an embodiment of the present invention. As shown in fig. 4, the interface data includes an SDTP protocol header and n XDR data, and the respective XDR data are compactly arranged. Before the omitted field backfill operation is carried out, interface data needs to be split to obtain each XDR data, so that each XDR data is independently subjected to the omitted field backfill.
Note that, in order to accommodate the backfill data with omitted fields, it is necessary to allocate enough memory space in advance for backfill. For XDR data, the length of each XDR data cannot exceed 64K, so that a fixed block memory space can be pre-allocated for backfilling.
Illustratively, sufficient memory space is pre-allocated for backfilling data prior to receiving the interface data. After receiving the interface data, the interface data is split according to the data transmission format to obtain a protocol header and a plurality of interface data segments. And reading the interface data segments one by one, and writing the interface data segments into the pre-allocated memory after reading the interface data segments each time.
For example, the interface data may be split according to a data transmission format of the interface data to obtain a plurality of XDR data. Specifically, after receiving the interface data, the content field other than the SDTP protocol header is extracted. The offset of each XDR data is determined according to the common header of each XDR data in the content field, and the content field in the interface data is split into a plurality of XDR data according to the offset of each XDR data. And sequentially caching each XDR data to a pre-allocated memory. Optionally, after the current XDR data is refilled, the next XDR data may be written to the pre-allocated memory in a manner of covering the current XDR data.
It should be noted that the execution sequence of step 340, step 320, and step 330 is not limited to the sequence listed in this embodiment, and step 340 may be executed first, then step 320 and step 330 may be executed, or step 340, step 320, and step 330 may be executed in parallel, which is not limited in the embodiment of the present invention.
Step 350, for each interface data segment, querying the global data interface table according to the interface type corresponding to the interface data segment to obtain a field to be backfilled, backfilling the field to be backfilled to the tail of the interface data segment, and updating a context information table in the data interface table corresponding to the interface type according to the field to be backfilled.
When the mobile network data is transmitted, some field data in some interfaces are not carried by the mobile network data, and the field data need to be acquired from other interfaces and then backfilled into the interface data of the interface. Specifically, the query of the global data interface table according to the interface type corresponding to the interface data segment to obtain the field to be backfilled may be: and determining a target field identifier for inquiring the global data interface table according to the interface type of the interface data to which the interface data segment belongs, inquiring the field table in the global data interface table according to the target field identifier, and taking the obtained field content as the field to be backfilled.
For example, the data of the N3 interface does not carry field data such as IMSI, PEI, MSISDN, RAT, etc. For the interface data of the N3 type, a global data interface table can be inquired to obtain the information of IMSI, PEI, MSISDN, RAT and the like in the interface data of N1N2, N4 and N11, wherein the N3 upf addr and N3 upf teid of the interface data are the same as the interface data of the N3 type, and the obtained information is used as a field to be backfilled to the interface data of the N3.
For another example, for N10 interface data, the data does not carry TAC and CELL ID fields, it may be determined according to the interface type N10 to query the global data interface table by using the IMSI field, obtain TAC, CELLID, and other information in the N1N2 interface data having the same IMSI field content, and backfill the TAC, CELLID, and other information into the N10 interface data as a field to be backfilled.
For another example, the wireless interface data does not carry information such as a mobile phone number, an IMSI, and a PEI. For the wireless interface data, a global data interface table can be inquired to obtain information of a mobile phone number, IMSI, PEI and the like in a signaling interface with the same amf _ ue _ ngap _ id, amf _ region _ id, amf _ set _ id, amf _ pointer _ id and cell _ id as the wireless interface data, and the obtained information is used as a field to be backfilled and is backfilled into the wireless interface data.
In the embodiment of the invention, for the condition that the backfill position of the omitted field is not reserved in the original interface data, the corresponding field to be backfilled is backfilled at the tail of each XDR data corresponding to the original interface data. As shown in FIG. 4, the fields to be backfilled are backfilled at the end of the original XDR data in the pre-allocated memory space.
And for each interface data segment included by the interface data, adding an index to each field to be backfilled, and updating a context information table in a data interface table corresponding to the interface type according to the index, the initial address and the offset of the field to be backfilled. Specifically, assuming that the original XDR data has 20 real fields, and it can be known by querying the field table that the original XDR data should have 40 fields, the remaining 20 fields are backfilled at the end of the original XDR data, and the sequence number of each backfilled field is used as an index, and the index, the start address, and the offset address of the field to be backfilled are stored in the context information table of the corresponding data interface table in an associated manner, so as to update the context information table, thereby avoiding the disorder of the fields.
It should be noted that, the original XDR data are sequentially written into the pre-allocated memory, and the backfilling operation is performed one by one.
As shown in fig. 4, after backfilling the field to be backfilled to the end of the original XDR data, a call ticket log saving operation is performed. For the call ticket log retention operation, the field table can be traversed to determine the number of fields of each XDR data, the corresponding number of indexes are determined from the context information table of the corresponding interface data table according to the number of the fields, and the context information of the fields or the context information of the sub-tables of the fields is read from the corresponding storage positions according to the starting address and the offset address corresponding to the indexes. And generating a call ticket log based on the field context information and the sub-table context information, and uploading the generated call ticket log to a web server for storage so that related personnel can log in the web server to inquire the call ticket log.
According to the technical scheme of the embodiment, a plurality of interface data segments are obtained by decomposing interface data, the interface data segments are sequentially cached in a pre-allocated memory, for the currently cached interface data segment, a global data interface table is inquired according to an interface type to obtain a field to be backfilled, the field to be backfilled is backfilled to the tail end of the interface data segment, and a context information table in the data interface table corresponding to the interface type is updated according to the field to be backfilled. In the embodiment, the corresponding field to be backfilled is backfilled to the end of the real field of the XDR data, and the position of the field to be backfilled does not need to be reserved in the interface data, so that the transmission flow of the interface data in the network can be reduced, and the utilization rate of the network bandwidth is also improved.
In a specific embodiment, a common configuration format is specified by analyzing the data characteristics of the existing mobile network interface data, and is used for describing the field format in all mobile network data interfaces. For example, the input data encoding types may be classified according to the kind of input data: v, LV, L1V, L4V, TV, TLV, etc. According to the actual output requirement, the decoding type of the output data is divided into: integer (integer), double precision floating point (double), ip, tbcd, string type (string), hex, etc. For the condition that 'the interface data comprises a plurality of groups of data and each group of data comprises a plurality of elements', regarding one group of data as a sub-table, and setting the name of the corresponding sub-table: NULL (no child table), child table name.
Assuming that 5G data interfaces are taken as an example, one data interface configuration file is generated for each data interface according to the 5G data interface specification. For example, fields in the data interface specification can be added to the corresponding data interface configuration file in a common configuration format by means of copy and paste, so as to convert the data interface specification into the corresponding data interface configuration file.
It should be noted that, according to the general configuration format, the data structure of the global data interface table is preset as follows, and all interface information is described by frames _ struct, which includes a plurality of structure arrays, and each structure array represents a type of data interface. The structure information of the data interface table of each type of data interface is described by frame _ struct, and the structure information comprises the name of the data interface, the number of fields of the data interface, the description of each field and the like. The configuration information of the fields of the data interface is described by files _ struct, and comprises information such as field names, types, lengths, tag values, decoding types, sub-table marks, sub-table names and sub-table ids. The context information of the fields of the data interface is described by filtered _ context _ structure, which is used for representing the analyzed result information and comprises information such as an omitted identifier, a data starting address, a data length, a data offset, the number of groups (also called sub-tables), the number of fields of each group (also called sub-tables), a sub-table id, a sub-table array structure and the like. The context information of the sub-table of the fields of the data interface is described by the sub-context structure, which contains the information of the omission flag, the data start address, the data length, the data offset, and the like.
Loading all data interface configuration files through a loading tool to generate a global data interface table, and realizing that a sub-table context information table of a field corresponds to information of a sub-context-structure array, a context information table of the field corresponds to information of a file-context-structure array, a field table corresponds to information of a file-structure array, a data interface table corresponds to information of a frame-structure array, and the global data interface table corresponds to information of the frame-structure data. When the composition program is initialized, the global data interface table is loaded to the memory.
When the interface data is received, the interface data is split to obtain a plurality of XDR data. And sequentially caching each XDR data to a pre-allocated memory space. Since the length of each piece of XDR data usually does not exceed 64K, a large enough memory space can be allocated in advance according to actual requirements for caching single XDR data. The interface type is determined from the common header carried in the XDR data. And searching the corresponding data interface table from the global data interface table based on the interface type. And resolving the cached XDR data according to a field table in the data interface table, and correspondingly writing the resolving result into a context information table of the field. If the XDR data includes at least one set of data, writing the parsed data elements into a sub-table context information table.
After the analysis is completed, if field backfilling is needed, whether the field to be backfilled exists in the original XDR data or not is detected. And if so, backfilling the field content corresponding to the field to be backfilled at the position of the field to be backfilled in the original XDR data. And if not, backfilling the field content of the field to be backfilled at the tail of the original XDR data. Therefore, the problem that the copy needs to be moved due to repeated insertion of data can be avoided, and the consumption of a CPU is reduced. For example, the field to be backfilled may be obtained from other types of interface data and backfilled to the end of the original XDR data. Mapping the backfilled field to a corresponding field context information table through an index relationship, and recording the context information such as the starting address, the data length, the offset and the like of the backfilled field. If a sub-table exists in the original XDR data, the index, start address, offset, etc. of the sub-table are written sequentially into the sub-table context information table. Therefore, when the call ticket log is stored, the corresponding field data can be inquired according to the index, and the problem of field disorder is solved.
Compared with the original scheme of reserving a certain space for interface data to be backfilled and refilling the data to the corresponding position, the technical scheme of the embodiment of the invention can reduce the transmission flow of the interface data in the network and improve the utilization rate of the network bandwidth.
Optionally, in order to be compatible with input data of a fixed length type, when a field to be refilled is acquired, whether a refill space is reserved in an original position of the field is determined according to an input type of original XDR data. If the field to be backfilled exists, the field to be backfilled is backfilled to the original position without updating the field identification (such as field id, offset, starting address, data length and the like) in the context information table; if the XDR data does not exist, backfilling the field to be backfilled at the tail end of the original XDR data, and updating the corresponding context information table according to the index, the initial address and the offset of the field to be backfilled.
Fig. 5 is a block diagram of an interface data processing apparatus, which may be implemented by software and/or hardware and is generally integrated in a computer device such as a composition server. The device can solve the problems of manpower development, debugging, testing and the like when a data interface is newly added by executing the interface data processing method provided by any embodiment of the invention. As shown in fig. 5, the apparatus includes:
a type determining module 510, configured to receive interface data, and determine an interface type of the interface data according to an interface type field in the interface data;
an interface table query module 520, configured to query a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, where the global data interface table is generated based on a data interface configuration file corresponding to an interface specification of each type of data interface;
a data parsing module 530, configured to parse the interface data according to the field table in the target data interface table, and write a parsing result into context information corresponding to the target data interface table in the global data interface table.
The embodiment of the invention provides a processing device of interface data, which determines the interface type through an interface type field contained in the interface data, inquires a pre-configured global data interface table based on the interface type to obtain a corresponding target data interface table, analyzes the interface data according to the field table in the target data interface table, and writes an analysis result into context information corresponding to the target data interface table in the global data interface table, thereby realizing a uniform interface data analysis function, avoiding the problems that the development, debugging, testing and the like are required to be invested with manpower for adding one data interface each time because of independent interface data analysis operation, and greatly reducing the development period, the debugging cost and the later maintenance cost.
Optionally, the apparatus further comprises:
an interface table loading module, configured to load a pre-generated global data interface table to a memory if initialization of a composition program is detected before determining an interface type of the interface data according to an interface type field in the interface data, where the global data interface table includes a data interface table, a field table, and a context information table;
the interface data table is used for describing the structural information of the data interface table;
the field table is used for describing configuration information of fields of the data interface;
the context information table is used for describing the context information of the fields of the data interface and/or describing the context information of the sub-tables of the fields of the data interface.
Optionally, the type determining module 510 is specifically configured to:
and analyzing an interface type field in the interface data to obtain interface type field content, and determining the interface type of the data interface corresponding to the interface data according to the interface type field content.
Optionally, the interface table query module 520 is specifically configured to:
inquiring each interface type identifier in a global data interface table according to the interface type to obtain an interface type identifier matched with the interface type;
and determining a target data interface table corresponding to the interface type according to the corresponding relation between the interface type identifier in the global data table and the interface data table.
Optionally, the data parsing module 530 is specifically configured to:
analyzing the interface data according to the configuration information of fields included in a field table in the target data interface table to obtain context information;
and writing the context information into a context information table corresponding to the target data interface table in the global data interface table.
Optionally, the data parsing module 530 is further specifically configured to:
under the condition that the interface data comprises at least one group of data with the element number larger than 1, analyzing the interface data according to configuration information of fields included in a field table in the target data interface table to obtain context information of the fields and context information of each sub-table;
writing the context information of the field into a field context information table corresponding to the target data interface table in the global data interface table;
and writing the context information of each sub-table into a sub-table context information table corresponding to the target data interface table in the global interface table.
Optionally, the apparatus further comprises:
the field backfill module is used for decomposing the interface data according to the data transmission format after receiving the interface data and respectively caching the obtained interface data segments in a pre-allocated memory; for each interface data segment, inquiring the global data interface table according to the interface type corresponding to the interface data segment to obtain a field to be backfilled, backfilling the field to be backfilled to the tail end of the interface data segment, and updating a context information table in the data interface table corresponding to the interface type according to the field to be backfilled.
Optionally, the field backfill module is further specifically configured to:
and for each interface data segment, adding an index to each field to be backfilled, and updating a context information table in a data interface table corresponding to the interface type according to the index, the initial address and the offset of the field to be backfilled.
The interface data processing device provided by the embodiment of the invention can execute the interface data processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present invention, as shown in fig. 6, the computer device includes a processor 60, a memory 61, and a communication component 62; the number of processors 60 in the computer device may be one or more, and one processor 60 is taken as an example in fig. 6; the processor 60, the memory 61, and the communication component 62 in the computer device may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 6.
The memory 61 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules (e.g., the type determining module 510, the interface table querying module 520, and the data parsing module 530) corresponding to the interface data processing method in the embodiment of the present invention. The processor 60 executes various functional applications of the computer device and data processing, i.e., implements the above-described processing method of the interface data, by executing software programs, instructions, and modules stored in the memory 61.
The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 61 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 61 may further include memory located remotely from the processor 60, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication component 62 is configured to enable a communication connection with the acquisition server.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for processing interface data, the method comprising:
receiving interface data, and determining the interface type of the interface data according to an interface type field in the interface data;
inquiring a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, wherein the global data interface table is generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces;
and analyzing the interface data according to a field table in the target data interface table, and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the interface data processing method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the processing apparatus for interface data, the included units and modules are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (11)

1. A method for processing interface data, comprising:
receiving interface data, and determining the interface type of the interface data according to an interface type field in the interface data;
inquiring a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, wherein the global data interface table is generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces;
and analyzing the interface data according to a field table in the target data interface table, and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
2. The method of claim 1, further comprising, before determining the interface type of the interface data according to an interface type field in the interface data:
if the initialization of the synthesis program is detected, loading a pre-generated global data interface table to a memory, wherein the global data interface table comprises a data interface table, a field table and a context information table;
the interface data table is used for describing the structural information of the data interface table;
the field table is used for describing configuration information of fields of the data interface;
the context information table is used for describing the context information of the fields of the data interface and/or describing the context information of the sub-tables of the fields of the data interface.
3. The method of claim 1, wherein determining the interface type of the interface data according to the interface type field in the interface data comprises:
and analyzing an interface type field in the interface data to obtain interface type field content, and determining the interface type of the data interface corresponding to the interface data according to the interface type field content.
4. The method according to claim 1, wherein the querying a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type includes:
inquiring each interface type identifier in a global data interface table according to the interface type to obtain an interface type identifier matched with the interface type;
and determining a target data interface table corresponding to the interface type according to the corresponding relation between the interface type identifier in the global data table and the interface data table.
5. The method according to claim 1, wherein the parsing the interface data according to the field table in the target data interface table, and writing a parsing result in context information corresponding to the target data interface table in the global data interface table includes:
analyzing the interface data according to the configuration information of fields included in a field table in the target data interface table to obtain context information;
and writing the context information into a context information table corresponding to the target data interface table in the global data interface table.
6. The method according to claim 5, wherein in a case that the interface data includes at least one group of data whose element number is greater than 1, the writing the context information into a context information table corresponding to the target data interface table in the global data interface table includes:
analyzing the interface data according to the configuration information of fields included in a field table in the target data interface table to obtain the context information of the fields and the context information of each sub-table;
writing the context information of the field into a field context information table corresponding to the target data interface table in the global data interface table;
and writing the context information of each sub-table into a sub-table context information table corresponding to the target data interface table in the global interface table.
7. The method of claim 1, after receiving the interface data, further comprising:
decomposing the interface data according to a data transmission format, and respectively caching the obtained interface data segments in a pre-allocated memory;
for each interface data segment, inquiring the global data interface table according to the interface type corresponding to the interface data segment to obtain a field to be backfilled, backfilling the field to be backfilled to the tail end of the interface data segment, and updating a context information table in the data interface table corresponding to the interface type according to the field to be backfilled.
8. The method according to claim 7, wherein the updating the context information table in the data interface table corresponding to the interface type according to the field to be backfilled comprises:
and for each interface data segment, adding an index to each field to be backfilled, and updating a context information table in a data interface table corresponding to the interface type according to the index, the initial address and the offset of the field to be backfilled.
9. An apparatus for processing interface data, comprising:
the type determining module is used for receiving interface data and determining the interface type of the interface data according to an interface type field in the interface data;
the interface table query module is used for querying a global data interface table according to the interface type to obtain a target data interface table corresponding to the interface type, wherein the global data interface table is generated based on data interface configuration files corresponding to interface specifications of various types of data interfaces;
and the data analysis module is used for analyzing the interface data according to the field table in the target data interface table and writing an analysis result into context information corresponding to the target data interface table in the global data interface table.
10. A computer device, characterized in that the computer device comprises:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a method of processing interface data as recited in any of claims 1-8.
11. A storage medium containing computer-executable instructions for performing the method of processing interface data according to any one of claims 1 to 8 when executed by a computer processor.
CN202110081553.4A 2021-01-21 2021-01-21 Interface data processing method and device, computer equipment and medium Active CN112765175B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110081553.4A CN112765175B (en) 2021-01-21 2021-01-21 Interface data processing method and device, computer equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110081553.4A CN112765175B (en) 2021-01-21 2021-01-21 Interface data processing method and device, computer equipment and medium

Publications (2)

Publication Number Publication Date
CN112765175A true CN112765175A (en) 2021-05-07
CN112765175B CN112765175B (en) 2024-05-24

Family

ID=75702188

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110081553.4A Active CN112765175B (en) 2021-01-21 2021-01-21 Interface data processing method and device, computer equipment and medium

Country Status (1)

Country Link
CN (1) CN112765175B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113535832A (en) * 2021-06-10 2021-10-22 杭州未名信科科技有限公司 Method and device for acquiring cloud service data, electronic equipment and medium
CN114125497A (en) * 2021-11-19 2022-03-01 中国电信集团系统集成有限责任公司 SFU architecture, media transmission method thereof and electronic equipment
CN116302041A (en) * 2023-04-06 2023-06-23 北京拙河科技有限公司 Optimization method and device for light field camera interface module

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1781280A (en) * 2003-03-14 2006-05-31 计算机联合思想公司 Relational model for management information in network devices
US7076727B1 (en) * 2000-08-16 2006-07-11 Sparta Systems, Inc. Configuring activities to perform operations on user-defined fields
US20070088625A1 (en) * 2005-10-18 2007-04-19 Fish Robert D Systems And Methods For Storing And Retrieving Goods And Services Information Using Parameter/Value Databases
CN102495556A (en) * 2011-12-13 2012-06-13 北京首钢自动化信息技术有限公司 Configurable apparatus data acquisition method
CN102904820A (en) * 2012-09-25 2013-01-30 上海证券交易所 Extensible traffic control data interaction method and system
CN103430621A (en) * 2011-02-19 2013-12-04 三星电子株式会社 Method and system of providing internet protocol (IP) data communication in a NFC peer to peer communication environment
CN106844643A (en) * 2017-01-13 2017-06-13 王洋 A kind of Database Dynamic generation method based on template engine
CN107908488A (en) * 2017-11-14 2018-04-13 中国银行股份有限公司 Message request interactive interfacing method, apparatus, computer equipment and storage medium
CN108920607A (en) * 2018-06-27 2018-11-30 中国建设银行股份有限公司 Field finds method, apparatus and electronic equipment
CN111459982A (en) * 2020-03-27 2020-07-28 中国平安人寿保险股份有限公司 Data query method and device, terminal device and storage medium
CN111614647A (en) * 2020-05-14 2020-09-01 中国银行股份有限公司 Message detection and analysis method and device
CN111708860A (en) * 2020-06-15 2020-09-25 北京优特捷信息技术有限公司 Information extraction method, device, equipment and storage medium
CN111797152A (en) * 2020-06-23 2020-10-20 北京自如信息科技有限公司 Data scene generation method and device and computer equipment
CN112148509A (en) * 2020-10-16 2020-12-29 腾讯科技(深圳)有限公司 Data processing method, device, server and computer readable storage medium
CN112241370A (en) * 2020-10-21 2021-01-19 网易(杭州)网络有限公司 Verification method, system and device for API (application program interface) interface class

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7076727B1 (en) * 2000-08-16 2006-07-11 Sparta Systems, Inc. Configuring activities to perform operations on user-defined fields
CN1781280A (en) * 2003-03-14 2006-05-31 计算机联合思想公司 Relational model for management information in network devices
US20070088625A1 (en) * 2005-10-18 2007-04-19 Fish Robert D Systems And Methods For Storing And Retrieving Goods And Services Information Using Parameter/Value Databases
CN103430621A (en) * 2011-02-19 2013-12-04 三星电子株式会社 Method and system of providing internet protocol (IP) data communication in a NFC peer to peer communication environment
CN102495556A (en) * 2011-12-13 2012-06-13 北京首钢自动化信息技术有限公司 Configurable apparatus data acquisition method
CN102904820A (en) * 2012-09-25 2013-01-30 上海证券交易所 Extensible traffic control data interaction method and system
CN106844643A (en) * 2017-01-13 2017-06-13 王洋 A kind of Database Dynamic generation method based on template engine
CN107908488A (en) * 2017-11-14 2018-04-13 中国银行股份有限公司 Message request interactive interfacing method, apparatus, computer equipment and storage medium
CN108920607A (en) * 2018-06-27 2018-11-30 中国建设银行股份有限公司 Field finds method, apparatus and electronic equipment
CN111459982A (en) * 2020-03-27 2020-07-28 中国平安人寿保险股份有限公司 Data query method and device, terminal device and storage medium
CN111614647A (en) * 2020-05-14 2020-09-01 中国银行股份有限公司 Message detection and analysis method and device
CN111708860A (en) * 2020-06-15 2020-09-25 北京优特捷信息技术有限公司 Information extraction method, device, equipment and storage medium
CN111797152A (en) * 2020-06-23 2020-10-20 北京自如信息科技有限公司 Data scene generation method and device and computer equipment
CN112148509A (en) * 2020-10-16 2020-12-29 腾讯科技(深圳)有限公司 Data processing method, device, server and computer readable storage medium
CN112241370A (en) * 2020-10-21 2021-01-19 网易(杭州)网络有限公司 Verification method, system and device for API (application program interface) interface class

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王业亮: "基于大数据的国际漫游多维度分析", 《中国移动通信集团湖南有限公司》, pages 24 - 26 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113535832A (en) * 2021-06-10 2021-10-22 杭州未名信科科技有限公司 Method and device for acquiring cloud service data, electronic equipment and medium
CN114125497A (en) * 2021-11-19 2022-03-01 中国电信集团系统集成有限责任公司 SFU architecture, media transmission method thereof and electronic equipment
CN114125497B (en) * 2021-11-19 2024-05-07 中电信数智科技有限公司 SFU architecture, media transmission method thereof and electronic equipment
CN116302041A (en) * 2023-04-06 2023-06-23 北京拙河科技有限公司 Optimization method and device for light field camera interface module
CN116302041B (en) * 2023-04-06 2023-11-21 北京拙河科技有限公司 Optimization method and device for light field camera interface module

Also Published As

Publication number Publication date
CN112765175B (en) 2024-05-24

Similar Documents

Publication Publication Date Title
CN112765175B (en) Interface data processing method and device, computer equipment and medium
CN111400127B (en) Service log monitoring method and device, storage medium and computer equipment
US20150304124A1 (en) Message Processing Method and Device
CN112650529B (en) System and method for configurable generation of mobile terminal APP codes
CN111966289A (en) Partition optimization method and system based on Kafka cluster
CN113094355A (en) Model instantiation method, device, equipment and medium based on domestic communication protocol
CN104836686B (en) A kind of method and device for the configuration file generating server
CN113596017A (en) Protocol analysis method, device, soft gateway and storage medium
KR20160103110A (en) Network element data access method and apparatus, and network management system
CN108429764A (en) A method of the data transmission based on proprietary protocol and parsing
CN109634908A (en) Data correlation method, data processing equipment and storage medium
CN106776372B (en) Emulation data access method and device based on FPGA
CN111443920B (en) Frame migration method and device
CN112416557A (en) Method and device for determining call relation, storage medium and electronic device
CN109521954B (en) Distribution network FTU fixed point file management method and device
CN106776285A (en) Webpage front-end adjustment method and device
US20110252039A1 (en) System, apparatus and method for processing content on a computing device
US20240005083A1 (en) Method and system for merging pdf files in a large batch
CN114024844B (en) Data scheduling method, data scheduling device and electronic equipment
CN116166640A (en) Real-time acquisition and management method and system for global navigation satellite observation data
US8270404B2 (en) System, method, and computer program product for improved distribution of data
CN112256714A (en) Data synchronization method and device, electronic equipment and computer readable medium
CN112417015A (en) Data distribution method and device, storage medium and electronic device
CN112965833B (en) Log processing method and device
CN204392305U (en) A kind of DTU data service system based on GPRS communication

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant