CN107318132B - Data distribution method and device in acquisition system - Google Patents
Data distribution method and device in acquisition system Download PDFInfo
- Publication number
- CN107318132B CN107318132B CN201610265342.5A CN201610265342A CN107318132B CN 107318132 B CN107318132 B CN 107318132B CN 201610265342 A CN201610265342 A CN 201610265342A CN 107318132 B CN107318132 B CN 107318132B
- Authority
- CN
- China
- Prior art keywords
- data
- synthesis server
- user information
- server
- data synthesis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/10—Flow control between communication endpoints
- H04W28/12—Flow control between communication endpoints using signalling between network elements
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
the invention discloses a data distribution method and device in an acquisition system. In the data distribution method, user information in the acquired data is acquired, and whether a synthesis server corresponding to the user information exists is inquired according to a corresponding relation table between the user information and the synthesis server; if yes, sending the acquired data to a synthesis server corresponding to the user information; otherwise, selecting a synthesis server for processing the acquired data, adding the corresponding relation between the user information and the selected synthesis server into the corresponding relation table, and sending the acquired data to the selected synthesis server.
Description
Technical Field
the present invention relates to the field of communications, and in particular, to a method and an apparatus for distributing data in an acquisition system.
Background
In an LTE (Long Term Evolution) signaling collection system, the system includes a data collection layer, a data decoding layer, and an application layer, as shown in fig. 1. The soft mining analysis server of the data acquisition layer receives data from a Uu/X2 interface output by SCA (traffic aggregation adapter) equipment, generates a single interface xDR (call/transaction detail record), and reports the single interface xDR to the data synthesis server of the data decoding layer. And the hard acquisition analysis server of the data acquisition layer receives data streams output by interface optical splitting links such as S1-MME/S6a/SGs/S11/S10/S5/S8 and the like, generates a single interface xDR and reports the single interface xDR to the data synthesis server of the data decoding layer. The soft mining analysis server acquires data in a soft mining mode, namely, the acquisition work is realized based on corresponding software when the signaling data passes through the base station; the hard acquisition analysis server acquires data in a hard acquisition mode, namely, the acquisition work is realized by adopting methods of adding light splitting equipment, copying, shunting and the like on a physical link. The data synthesis server receives the single-interface xDR reported by the soft mining analysis server and the hard mining analysis server, correlates the soft mining with the hard mining analysis server, and outputs the synthesized xDR to the application layer.
According to the processing capacity of the soft mining analysis server, each soft mining analysis server can access data reported by a plurality of SCAs, the soft mining analysis server can take the SCAs as the minimum granularity of data acquisition, and each SCA can access the data reported by 500 base stations at the maximum. The hard acquisition analysis server can be accessed to one or more optical splitting links, and a plurality of S1-MME interface links or data transmitted on other interface links are converged on one optical splitting link.
At present, a data distribution and data synthesis server reported by a soft acquisition or hard acquisition analysis server is based on a corresponding relationship between a base station and a data synthesis server ID (Identification). The data distribution method has the problems of low data correlation and backfill accuracy.
Disclosure of Invention
the embodiment of the invention provides a data distribution method and device in an acquisition system, which are used for realizing data distribution and data synthesis servers according to user information.
The data distribution method in the acquisition system provided by the embodiment of the invention comprises the following steps:
Acquiring user information in the acquired data;
inquiring whether a data synthesis server corresponding to the user information exists according to a corresponding relation table between the user information and the data synthesis server;
If the user information exists, the acquired data is sent to a data synthesis server corresponding to the user information; otherwise, selecting a data synthesis server for processing the acquired data, adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table, and sending the acquired data to the selected data synthesis server.
Preferably, the method further comprises:
Counting data traffic of M data synthesis servers according to a set period, wherein M is an integer greater than 1;
If the difference value between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server is greater than a preset threshold in a statistical period, modifying the corresponding relation between the N pieces of user information and the first data synthesis server into the corresponding relation between the N pieces of user information and the second data synthesis server in the corresponding relation table; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
Preferably, the N users corresponding to the N user information are the first N users arranged according to the data traffic from large to small among the users corresponding to the first data synthesis server.
Preferably, the sum of the data traffic of N users corresponding to the N pieces of user information is greater than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server, and the sum of the data traffic of any N-1 users in the N users is less than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server.
Specifically, the selecting a data synthesis server for processing the acquired data includes:
Selecting a number of data synthesis servers for processing the acquired data according to the following formula:
L＝A mod B
where L denotes the number of data composition servers, a denotes the number of users, and B denotes the number of data composition servers.
specifically, the correspondence table between the user and the data synthesis server includes: the corresponding relation between the first type user identification and the data synthesis server identification;
According to a corresponding relation table between user information and a data synthesis server, inquiring whether the data synthesis server corresponding to the user information exists or not, wherein the method comprises the following steps:
if the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification;
And inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
specifically, the user information includes one or more of the following: international mobile subscriber identity IMSI, international mobile equipment identity IMEI, mobile subscriber number MSISDN, mobility management entity temporary mobile subscriber identity M-TMSI, MME _ UE _ S1AP _ ID.
the data distribution device in the acquisition system provided by the embodiment of the invention comprises:
the acquisition module is used for acquiring user information in the acquired data;
The query module is used for querying whether the data synthesis server corresponding to the user information exists or not according to a corresponding relation table between the user information and the data synthesis server;
the sending module is used for sending the acquired data to the data synthesis server corresponding to the user information if the query result of the query module indicates that the data synthesis server corresponding to the user information exists;
and the selection module is used for selecting a data synthesis server for processing the acquired data if the query result of the query module indicates that the data synthesis server corresponding to the user information does not exist, adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table, and indicating the sending module to send the acquired data to the selected data synthesis server.
Preferably, the apparatus further comprises:
The statistical module is used for carrying out statistics on data traffic of M data synthesis servers according to a set period, wherein M is an integer greater than 1;
A modification module, configured to modify, in the correspondence table, a correspondence between the N pieces of user information and the first data synthesis server to a correspondence between the N pieces of user information and the second data synthesis server if, in a statistical period, a difference between a data traffic of the first data synthesis server and a data traffic of the second data synthesis server is greater than a preset threshold; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
Preferably, the N users corresponding to the N user information are the first N users arranged according to the data traffic from large to small among the users corresponding to the first data synthesis server.
Preferably, the sum of the data traffic of N users corresponding to the N pieces of user information is greater than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server, and the sum of the data traffic of any N-1 users in the N users is less than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server.
specifically, the selection module is specifically configured to:
selecting a number of data synthesis servers for processing the acquired data according to the following formula:
L＝A mod B
Where L denotes the number of data composition servers, a denotes the number of users, and B denotes the number of data composition servers.
Specifically, the correspondence table between the user and the data synthesis server includes: the corresponding relation between the first type user identification and the data synthesis server identification;
the query module is specifically configured to:
if the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification;
and inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
Specifically, the user information includes one or more of the following: international mobile subscriber identity IMSI, international mobile equipment identity IMEI, mobile subscriber number MSISDN, mobility management entity temporary mobile subscriber identity M-TMSI, mobility management entity S1 application user equipment identity MME _ UE _ S1AP _ ID.
in the above embodiment of the present invention, according to the user information in the collected data and the correspondence table between the user information and the data synthesis server, the data synthesis server corresponding to the user information is queried, and the collected data is sent to the corresponding data synthesis server; and if the corresponding relation table between the user information and the data synthesis server does not have the data synthesis server corresponding to the user information, selecting the data synthesis server for processing the acquired data, and adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table. Because the data synthesis server is distributed for the collected data according to the user information, the data of the same user is distributed to the same data synthesis server for association, backfill and synthesis, and the accuracy of data association and backfill is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic diagram of an architecture of an LTE signaling acquisition system in the prior art;
fig. 2 is a schematic flow chart of a data distribution method in the acquisition system according to an embodiment of the present invention;
fig. 3 is a second schematic flow chart of a data distribution method in the acquisition system according to the embodiment of the present invention;
fig. 4 is a schematic flow chart of a data offloading method according to an embodiment of the present invention;
Fig. 5 is a schematic structural diagram of a data distribution device in the acquisition system according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a data offloading device according to an embodiment of the present invention;
Fig. 7 is a schematic structural diagram of a data distribution device in another acquisition system according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another data offloading device according to an embodiment of the present invention.
Detailed Description
in order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a data distribution method in an acquisition system, which is used for realizing data distribution and data synthesis servers according to user information.
the method is applicable to the signaling collection system architecture shown in fig. 1, and can be executed by a data synthesis server in the architecture. Specifically, the data distribution apparatus that executes the embodiment of the present invention may be integrated in a data synthesis server, determine a data synthesis server that processes received data by accessing a correspondence table between user information and the data synthesis server, and transmit the received data to the data synthesis server corresponding thereto, where the correspondence table between the user information and the data synthesis server may be stored in any one data synthesis server or in a plurality of data synthesis servers. The data distribution device executing the embodiment of the present invention may be an independent data synthesis server, and the received data are all distributed to other data synthesis servers through the data synthesis server for processing.
referring to fig. 2, a schematic flow chart of a data distribution method in the acquisition system according to the embodiment of the present invention is shown. As shown in fig. 2, the method comprises the steps of:
step 201: the data distribution device acquires user information in the acquired data.
the Subscriber information includes a fixed Identity, for example, one or more of an IMSI (International Mobile Subscriber Identity), an IMEI (International Mobile Equipment Identity), an MSISDN (Mobile Subscriber Number), and the like. The user information may also include Temporary identities, such as one or more of M-TMSI (MME-Temporary Mobile Subscriber Identity), MME _ UE _ S1AP _ ID (user Temporary Identity of the mobility management entity at S1-MME reference point), and the like. The types of user information contained in the data collected from different interfaces are also different, the user information contained in the data of each interface is shown in table 1, some interfaces only contain fixed identification marks, some interfaces only contain temporary identification marks, and some interfaces contain both fixed identification marks and temporary identification marks.
TABLE 1
the square root in table 1 indicates that the data transmitted in the interface corresponding to the row includes the subscriber identity corresponding to the column, for example, the data transmitted in the Uu interface includes MME _ UE _ S1AP _ ID, and the data transmitted in the S1-MME interface includes MME _ UE _ S1AP _ ID, IMSI, IMEI, and M-TMSI.
Step 202: the data distribution device inquires whether a data synthesis server corresponding to the user information exists according to a corresponding relation table between the user information and the data synthesis server. If yes, go to step 203, otherwise, go to step 204.
Since the user information included in each interface data is different, when the correspondence table between the user information and the data synthesis server is searched, the data synthesis server corresponding to the input user information can be searched regardless of the type of user information input.
Preferably, the above technical problem can be solved by: the correspondence table between the user information and the data synthesis server includes a correspondence table between a first type of user identifier and the data synthesis server, for example, IMSI may be used as the first type of user identifier, other types of user identifiers may be used as the second type of user identifier, and the first type of user identifier and the second type of user identifier may be predefined in specific implementation. If the user information acquired by the data distribution device contains a first type user identifier, inquiring whether a data synthesis server corresponding to the user information exists or not according to a corresponding relation table between the first type user identifier and the data synthesis server; if the user information acquired by the data distribution device is the second type user identification, determining the first type user identification corresponding to the acquired second type user identification according to the corresponding relation between the first type user identification and the second type user identification, and then inquiring whether the data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
Taking the IMSI as the first type subscriber identity and the other subscriber identities as the second type subscriber identity as an example, the correspondence table between the subscriber information and the data synthesis server is the correspondence table between the IMSI and the data synthesis server, and the correspondence between the first type subscriber identity and the second type subscriber identity can be represented by the following correspondence table:
Table of correspondence between MME _ UE _ S1AP _ ID and IMSI
IMEI, MSISDN and IMSI
table of correspondence between M-TMSI and IMSI
Of course, the correspondence between the first type user identifier and the second type user identifier may be represented by the three correspondence tables, but is not limited thereto, and may also be represented by only one correspondence table or a plurality of correspondence tables in other forms. In addition, the method for dividing the first type of user identifier and the second type of user identifier is also only an embodiment provided by the present invention, and is not limited thereto.
Step 203: the data distribution device transmits the acquired data to the data synthesis server corresponding to the user information.
Step 204: the data distribution device selects a data synthesis server for processing the acquired data, adds the correspondence between the user information and the selected data synthesis server to the correspondence table, and transmits the acquired data to the selected data synthesis server.
For a new user who cannot find a corresponding data synthesis server in the correspondence table between the user information and the data synthesis server according to the user information, a data synthesis server should be selected for the user to process the data of the user. When selecting a data composition server for a new user, the selection can be made according to different methods and according to different principles.
One embodiment of the present invention provides a selection method for selecting a data synthesis server for a new user according to a principle of user number balance. Specifically, after acquiring user information in the acquired data, if the data synthesis server corresponding to the user information cannot be queried, the data distribution device determines the total number of the current users, and then calculates the data synthesis server for processing the data of the user according to the following formula:
L＝AmodB(1)
Where L denotes the number of data synthesis servers that process data for the user, a denotes the total number of previous users, and B denotes the number of data synthesis servers.
In the method, the data synthesis servers are selected according to the principle of user data balance, and the aim is to basically balance the load condition of each data synthesis server. However, due to different liveness among users, the amount of reported data may be greatly different, which may cause a large difference in load condition of the data composition server. Therefore, the embodiment of the invention also provides a method for balancing the load condition of the data synthesis server so as to improve the utilization rate of system resources. The method process can be executed according to a set period, and in a set period, the flow diagram is shown in fig. 3, and the method includes the following steps:
Step 301: and counting the data traffic of the plurality of data synthesis servers according to a set period. In practical application, data traffic of all data synthesis servers in the system architecture is often counted.
Step 302: judging whether the difference value between the data flow of the first data synthesis server and the data flow of the second data synthesis server is greater than a preset threshold or not in a statistical period; the first data synthesis server is the data synthesis server with the largest data flow in the statistical period, the second data synthesis server is the data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1. If yes, go to step 303, otherwise, do not operate in this period.
step 303: in the corresponding relation table between the user information and the data synthesis server, the corresponding relation between the N pieces of user information and the first data synthesis server is modified into the corresponding relation between the N pieces of user information and the second data synthesis server.
Preferably, the N users corresponding to the N user information are the first N users arranged according to the data traffic from large to small among the users corresponding to the first data synthesis server. The corresponding relation between the N users with larger data volume and the data synthesis server with the largest data flow is modified into the corresponding relation with the data synthesis server with the smallest data flow, and the effect of balancing the load of the data synthesis server can be achieved under the condition that the corresponding relation table between the user information and the data synthesis server is slightly modified.
Preferably, the sum of the data traffic of the N users corresponding to the N pieces of user information is greater than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server, and the sum of the data traffic of any N-1 users in the N users is less than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server. The data synthesis servers corresponding to the N pieces of user information which meet the conditions are modified, so that the load conditions of the data synthesis server with the maximum data flow and the data synthesis server with the minimum data flow can be balanced most.
In the above embodiment of the present invention, according to the user information in the collected data and the correspondence table between the user information and the data synthesis server, the data synthesis server corresponding to the user information is queried, and the collected data is sent to the corresponding data synthesis server; and if the corresponding relation table between the user information and the data synthesis server does not have the data synthesis server corresponding to the user information, selecting the data synthesis server for processing the acquired data, and adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table. Because the data synthesis server is distributed for the collected data according to the user information, the data of the same user is distributed to the same data synthesis server for association, backfill and synthesis, and the accuracy of data association and backfill is improved. And because the data traffic of each data synthesis server is counted according to the set period, the corresponding relation table between the user information and the data synthesis servers is modified when the load is unbalanced, so as to balance the load condition of each data synthesis server.
In addition, the embodiment of the invention also provides a data distribution method in the acquisition system, which can be applied to the convergence splitter to solve the problem of data distribution when the data flow in the optical distribution link is greater than the processing capacity of the hard acquisition analysis server.
referring to fig. 4, a schematic flow chart of a data offloading method in an acquisition system according to an embodiment of the present invention is shown, where as shown in the drawing, the method includes the following steps:
step 401: the convergence splitter obtains the data flow transmitted by the optical splitting link, wherein K data flows are converged in the optical splitting link, and K is an integer greater than or equal to 1.
in practical applications, one or more optical splitter links may be connected to one aggregation splitter, one optical splitter link transmits data reported by one or more interfaces, and the data reported by each interface includes data reported by a plurality of interface links. For example, a drop link may only include data of the S1-MME interface, data of the S1-MME interface includes data reported by a plurality of S1-MME interface links, each S1-MME interface link represents a link between a base station and an MME, and data transmitted on each S1-MME interface link is configured as a data stream.
step 402: the convergence splitter distributes the data transmitted by the optical splitting link to a hard acquisition analysis server according to the data traffic of the optical splitting link and the data traffic which can be received by the hard acquisition analysis server; if the flow of one optical splitting link is larger than the data flow which can be accepted by one hard acquisition analysis server, the data transmitted by the optical splitting link is sent to a plurality of hard acquisition analysis servers, and the data flow on the same interface link is sent to the same hard acquisition analysis server.
Preferably, when the traffic of one optical splitter link is greater than the data traffic that can be received by one hard acquisition analysis server, the convergence splitter may obtain information for identifying data streams in each data stream transmitted by the optical splitter link; and sending all data streams transmitted by the optical splitting link to a plurality of hard acquisition analysis servers according to the information for identifying the data streams in each data stream, and sending the data streams on the same interface link to the same hard acquisition analysis server. Although the data transmitted in one optical splitting link is sent to different hard acquisition and analysis servers, the data of the data stream on the same interface link is sent to the same hard acquisition and analysis server, so that the data of the same service of the same user can be sent to the same hard acquisition and analysis server, and the operations of the hard acquisition and analysis server for synthesizing the received data by the single interface xDR and the like cannot be influenced.
Specifically, the information for identifying the data stream in the data stream on one interface link includes: an IP quintuple of the data stream, the IP quintuple comprising: source IP address, destination IP address, source port, destination port, transport layer protocol.
Further, if the flow of one or more optical splitting links is less than or equal to the data flow which can be accepted by one hard acquisition analysis server, the data transmitted by the one or more optical splitting links is sent to one hard acquisition analysis server.
In the above embodiment of the present invention, when data in the optical splitting link is distributed, if the traffic of one optical splitting link is greater than the data traffic that can be accommodated, the data transmitted by the optical splitting link is sent to a plurality of hard acquisition and analysis servers, and the data stream on the same interface link is sent to the same hard acquisition and analysis server. The problem that if the flow on the optical splitting link is large, the data flow exceeding the acceptable data flow is caused, so that overload operation is caused, and the efficiency of the system is reduced is solved.
Based on the same technical concept, the embodiment of the present invention further provides a device for acquiring data distribution in a system, as shown in fig. 5, the device includes an obtaining module 501, a querying module 502, a sending module 503 and a selecting module 504, and further, the device may further include a statistics module 505 and a modification module 506.
specifically, the obtaining module 501 is configured to obtain user information in the collected data.
The query module 502 is configured to query whether a data synthesis server corresponding to the user information exists according to a correspondence table between the user information and the data synthesis server.
the sending module 503 is configured to send the acquired data to the data synthesis server corresponding to the user information if the query result of the query module indicates that the data synthesis server corresponding to the user information exists.
The selecting module 504 is configured to select a data synthesis server for processing the acquired data if the query result of the querying module indicates that the data synthesis server corresponding to the user information does not exist, add the correspondence between the user information and the selected data synthesis server to the correspondence table, and instruct the sending module to send the acquired data to the selected data synthesis server.
The statistical module 505 is configured to perform statistics on data traffic of M data synthesis servers according to a set period, where M is an integer greater than 1.
The modifying module 506 is configured to modify, in the correspondence table, a correspondence between the N pieces of user information and the first data synthesis server into a correspondence between the N pieces of user information and the second data synthesis server if a difference between a data traffic of the first data synthesis server and a data traffic of the second data synthesis server is greater than a preset threshold in a statistical period; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
preferably, the N users corresponding to the N user information are the first N users arranged according to the data traffic from large to small among the users corresponding to the first data synthesis server.
preferably, a sum of data traffic of N users corresponding to the N pieces of user information is greater than or equal to one half of a difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server, and a sum of data traffic of any N-1 users in the N users is less than or equal to one half of a difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server.
specifically, the selecting module 504 may select a number of a data synthesis server for processing the acquired data according to formula (1).
specifically, the correspondence table between the user and the data synthesis server includes: and the first type user identification and the data synthesis server identification are in corresponding relation. The query module 502 is specifically configured to: if the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification; and inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
Specifically, the user information includes one or more of IMSI, IMEI, MSISDN, M-TMSI, and MME _ UE _ S1AP _ ID.
In the above embodiment of the present invention, according to the user information in the collected data and the correspondence table between the user information and the data synthesis server, the data synthesis server corresponding to the user information is queried, and the collected data is sent to the corresponding data synthesis server; and if the corresponding relation table between the user information and the data synthesis server does not have the data synthesis server corresponding to the user information, selecting the data synthesis server for processing the acquired data, and adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table. Because the data synthesis server is distributed for the collected data according to the user information, the data of the same user is distributed to the same data synthesis server for association, backfill and synthesis, and the accuracy of data association and backfill is improved. And because the data traffic of each data synthesis server is counted according to the set period, the corresponding relation table between the user information and the data synthesis servers is modified when the load is unbalanced, so as to balance the load condition of each data synthesis server.
Based on the same technical concept, an embodiment of the present invention further provides a data offloading device, as shown in fig. 6, the device specifically includes an obtaining module 601 and a distributing module 602.
The obtaining module 601 is configured to obtain a data traffic size transmitted by an optical splitter link, where K data streams are aggregated in the optical splitter link, where K is an integer greater than or equal to 1.
The distribution module 602 is configured to distribute data transmitted by the optical splitter link to a hard acquisition analysis server according to the size of data traffic of the optical splitter link and data traffic that can be received by the hard acquisition analysis server; if the flow of one optical splitting link is larger than the data flow which can be accepted by one hard acquisition analysis server, the data transmitted by the optical splitting link is sent to a plurality of hard acquisition analysis servers, and the data flow on the same interface link is sent to the same hard acquisition analysis server.
Preferably, the distribution module 602 is specifically configured to obtain information used for identifying a data stream in each data stream transmitted by the optical drop link; and sending all data streams transmitted by the optical splitting link to a plurality of hard acquisition analysis servers according to the information for identifying the data streams in each data stream, and sending the data streams on the same interface link to the same hard acquisition analysis server.
Specifically, the information for identifying the data stream in the data stream on one interface link includes: an IP quintuple of the data stream, the IP quintuple comprising: source IP address, destination IP address, source port, destination port, transport layer protocol.
Preferably, the distribution module 602 is further configured to: and if the flow of one or more optical splitting links is less than or equal to the data flow which can be accepted by one hard acquisition analysis server, sending the data transmitted by the one or more optical splitting links to one hard acquisition analysis server.
In the above embodiment of the present invention, when data in the optical splitting link is distributed, if the traffic of one optical splitting link is greater than the data traffic that can be accommodated, the data transmitted by the optical splitting link is sent to a plurality of hard acquisition and analysis servers, and the data stream on the same interface link is sent to the same hard acquisition and analysis server. The problem that if the flow on the optical splitting link is large, the data flow exceeding the acceptable data flow is caused, so that overload operation is caused, and the efficiency of the system is reduced is solved.
Based on the same technical concept, the embodiment of the invention also provides a data distribution device in the acquisition system, and the device can realize the process of distributing the data synthesis server to the data.
referring to fig. 7, a schematic structural diagram of an apparatus according to an embodiment of the present invention is provided, where the apparatus may include: a processor 701, a memory 702, a transceiver 703, and a bus interface.
the processor 701 is responsible for managing the bus architecture and general processing, and the memory 702 may store data used by the processor 701 in performing operations. The transceiver 703 is used for receiving and transmitting data under the control of the processor 701.
the bus architecture may include any number of interconnected buses and bridges, with one or more processors, represented by processor 701, and various circuits, represented by memory 702, being linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface. The transceiver 703 may be a number of elements, including a transmitter and a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 701 is responsible for managing the bus architecture and general processing, and the memory 702 may store data used by the processor 701 in performing operations.
The process of allocating a data composition server for data disclosed in the embodiments of the present invention may be applied to the processor 701, or implemented by the processor 701. In implementation, the steps of the process of assigning a data composition server to data may be performed by instructions in the form of hardware integrated logic circuits or software in the processor 701. The processor 701 may be a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like that may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702, and completes the steps of the data distribution data synthesis server flow in combination with the hardware thereof.
Specifically, the processor 701, configured to read the program in the memory 702, executes the following processes:
Acquiring user information in the acquired data;
Inquiring whether a data synthesis server corresponding to the user information exists according to a corresponding relation table between the user information and the data synthesis server;
If the user information exists, the acquired data is sent to a data synthesis server corresponding to the user information; otherwise, selecting a data synthesis server for processing the acquired data, adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table, and sending the acquired data to the selected data synthesis server.
Specifically, the processor 701 selects the number of the data synthesis server for processing the acquired data according to the above formula (1).
Preferably, the correspondence table between the user and the data synthesis server includes: and the first type user identification and the data synthesis server identification are in corresponding relation.
the processor 701 is specifically configured to:
If the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification;
And inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
Preferably, the processor 701 is further configured to:
Counting data traffic of M data synthesis servers according to a set period, wherein M is an integer greater than 1; if the difference value between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server is greater than a preset threshold in a statistical period, modifying the corresponding relation between the N pieces of user information and the first data synthesis server into the corresponding relation between the N pieces of user information and the second data synthesis server in the corresponding relation table; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
Preferably, the N users corresponding to the N user information are the first N users arranged according to the data traffic from large to small among the users corresponding to the first data synthesis server.
Preferably, the sum of the data traffic of N users corresponding to the N pieces of user information is greater than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server, and the sum of the data traffic of any N-1 users in the N users is less than or equal to one half of the difference between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server.
specifically, the user information includes one or more of the following: international mobile subscriber identity IMSI, international mobile equipment identity IMEI, mobile subscriber number MSISDN, mobility management entity temporary mobile subscriber identity M-TMSI, mobility management entity S1 application user equipment identity MME _ UE _ S1AP _ ID.
Based on the same technical concept, the embodiment of the invention also provides a data distribution device, and the device can realize the data distribution process in the embodiment of the invention.
Referring to fig. 8, a schematic structural diagram of a data offloading device according to an embodiment of the present invention is provided, where the device may include: a processor 801, a memory 802, a transceiver 803, and a bus interface.
The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 may store data used by the processor 801 in performing operations. The transceiver 803 is used for receiving and transmitting data under the control of the processor 801.
The bus architecture may include any number of interconnected buses and bridges, with one or more processors, represented by the processor 801, and various circuits, represented by the memory 802, being linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface. The transceiver 803 may be a plurality of elements, i.e., including a transmitter and a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 may store data used by the processor 801 in performing operations.
The flow of data splitting disclosed in the embodiment of the present invention may be applied to the processor 801, or implemented by the processor 801. In implementation, the steps of the data flow may be implemented by hardware integrated logic circuits or instructions in software in the processor 801. The processor 801 may be a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like that implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 802, and the processor 801 reads information in the memory 802, and completes the steps of the data offloading process in combination with hardware thereof.
Specifically, the processor 801, which is configured to read the program in the memory 802, executes the following processes:
acquiring the data traffic transmitted by an optical splitting link, wherein K data streams are converged in the optical splitting link, and K is an integer greater than or equal to 1;
distributing the data transmitted by the optical splitting link to a hard acquisition analysis server according to the data traffic of the optical splitting link and the data traffic which can be accepted by the hard acquisition analysis server; if the flow of one optical splitting link is larger than the data flow which can be accepted by one hard acquisition analysis server, the data transmitted by the optical splitting link is sent to a plurality of hard acquisition analysis servers, and the data flow on the same interface link is sent to the same hard acquisition analysis server.
Specifically, the processor 801 is specifically configured to obtain information used for identifying data streams in each data stream transmitted by the optical drop link; and sending all the data streams transmitted by the optical splitting link to a plurality of hard acquisition analysis servers according to the information for identifying the data streams in each data stream, wherein the data streams on the same interface link are sent to the same hard acquisition analysis server.
Specifically, the information for identifying the data stream in the data stream on one interface link includes: an IP quintuple of the data stream, the IP quintuple comprising: source IP address, destination IP address, source port, destination port, transport layer protocol.
optionally, the processor 801 is further configured to send data transmitted by the X optical division links to a hard acquisition analysis server if traffic of the X optical division links is less than or equal to data traffic that can be received by the hard acquisition analysis server; wherein X is an integer of 1 or more.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
it will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (14)
1. A data distribution method in an acquisition system is characterized by comprising the following steps:
Acquiring user information in the acquired data;
inquiring whether a data synthesis server corresponding to the user information exists according to a corresponding relation table between the user information and the data synthesis server;
If the user information exists, the acquired data is sent to a data synthesis server corresponding to the user information; otherwise, selecting a data synthesis server for processing the acquired data, adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table, and sending the acquired data to the selected data synthesis server.
2. The method of claim 1, further comprising:
counting data traffic of M data synthesis servers according to a set period, wherein M is an integer greater than 1;
If the difference value between the data traffic of the first data synthesis server and the data traffic of the second data synthesis server is greater than a preset threshold in a statistical period, modifying the corresponding relation between the N pieces of user information and the first data synthesis server into the corresponding relation between the N pieces of user information and the second data synthesis server in the corresponding relation table; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
3. The method according to claim 2, wherein the N users corresponding to the N user information are the first N users arranged from large to small according to data traffic among the users corresponding to the first data synthesis server.
4. The method of claim 2, wherein a sum of data traffic of N users corresponding to the N user information is greater than or equal to one-half of a difference between data traffic of the first data composition server and data traffic of the second data composition server, and a sum of data traffic of any N-1 users of the N users is less than or equal to one-half of a difference between data traffic of the first data composition server and data traffic of the second data composition server.
5. The method of claim 1, wherein the selecting a data synthesis server for processing the acquired data comprises:
Selecting a number of data synthesis servers for processing the acquired data according to the following formula:
L＝A mod B
where L denotes the number of data composition servers, a denotes the number of users, and B denotes the number of data composition servers.
6. The method of claim 1, wherein the correspondence table between the user and the data composition server comprises: the corresponding relation between the first type user identification and the data synthesis server identification;
According to a corresponding relation table between user information and a data synthesis server, inquiring whether the data synthesis server corresponding to the user information exists or not, wherein the method comprises the following steps:
If the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification;
and inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
7. The method of any one of claims 1 to 6, wherein the user information comprises one or more of: international mobile subscriber identity IMSI, international mobile equipment identity IMEI, mobile subscriber number MSISDN, mobility management entity temporary mobile subscriber identity M-TMSI, MME _ UE _ S1AP _ ID.
8. An apparatus for data distribution in an acquisition system, comprising:
the acquisition module is used for acquiring user information in the acquired data;
The query module is used for querying whether the data synthesis server corresponding to the user information exists or not according to a corresponding relation table between the user information and the data synthesis server;
The sending module is used for sending the acquired data to the data synthesis server corresponding to the user information if the query result of the query module indicates that the data synthesis server corresponding to the user information exists;
And the selection module is used for selecting a data synthesis server for processing the acquired data if the query result of the query module indicates that the data synthesis server corresponding to the user information does not exist, adding the corresponding relation between the user information and the selected data synthesis server into the corresponding relation table, and indicating the sending module to send the acquired data to the selected data synthesis server.
9. the apparatus of claim 8, further comprising:
The statistical module is used for carrying out statistics on data traffic of M data synthesis servers according to a set period, wherein M is an integer greater than 1;
A modification module, configured to modify, in the correspondence table, a correspondence between the N pieces of user information and the first data synthesis server to a correspondence between the N pieces of user information and the second data synthesis server if, in a statistical period, a difference between a data traffic of the first data synthesis server and a data traffic of the second data synthesis server is greater than a preset threshold; the first data synthesis server is a data synthesis server with the largest data flow in the statistical period, the second data synthesis server is a data synthesis server with the smallest data flow in the statistical period, and N is an integer greater than or equal to 1.
10. The apparatus according to claim 9, wherein the N users corresponding to the N user information are the first N users arranged according to data traffic from top to bottom among the users corresponding to the first data composition server.
11. The apparatus of claim 9, wherein a sum of data traffic of N users corresponding to the N user information is greater than or equal to one-half of a difference between data traffic of the first data composition server and data traffic of the second data composition server, and a sum of data traffic of any N-1 users of the N users is less than or equal to one-half of a difference between data traffic of the first data composition server and data traffic of the second data composition server.
12. the apparatus of claim 8, wherein the selection module is specifically configured to:
selecting a number of data synthesis servers for processing the acquired data according to the following formula:
L＝A mod B
where L denotes the number of data composition servers, a denotes the number of users, and B denotes the number of data composition servers.
13. The apparatus of claim 8, wherein the correspondence table between the user and the data composition server comprises: the corresponding relation between the first type user identification and the data synthesis server identification;
The query module is specifically configured to:
If the user information in the acquired data is a second type user identification, determining a first type user identification corresponding to the second type user identification according to the corresponding relation between the first type user identification and the second type user identification;
and inquiring whether a data synthesis server corresponding to the user information exists according to the determined corresponding relation table between the first type user identification and the data synthesis server.
14. The apparatus of any one of claims 8 to 13, wherein the user information comprises one or more of: international mobile subscriber identity IMSI, international mobile equipment identity IMEI, mobile subscriber number MSISDN, mobility management entity temporary mobile subscriber identity M-TMSI, mobility management entity S1 application user equipment identity MME _ UE _ S1AP _ ID.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610265342.5A CN107318132B (en) | 2016-04-26 | 2016-04-26 | Data distribution method and device in acquisition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610265342.5A CN107318132B (en) | 2016-04-26 | 2016-04-26 | Data distribution method and device in acquisition system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107318132A CN107318132A (en) | 2017-11-03 |
| CN107318132B true CN107318132B (en) | 2019-12-06 |
Family
ID=60184372
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610265342.5A Active CN107318132B (en) | 2016-04-26 | 2016-04-26 | Data distribution method and device in acquisition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107318132B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111372209B (en) * | 2018-12-06 | 2022-04-08 | 中国移动通信集团辽宁有限公司 | Signaling data processing method, device, equipment and medium |
| CN112291185B (en) * | 2019-07-24 | 2022-11-22 | 中国移动通信集团贵州有限公司 | Method and device for collecting network data |
| CN113079524B (en) * | 2020-01-06 | 2022-04-22 | 大唐移动通信设备有限公司 | Signaling tracking method based on signaling soft acquisition, base station and readable storage medium |
| CN112330360B (en) * | 2020-11-04 | 2023-09-01 | 中国联合网络通信集团有限公司 | Advertisement pushing method and system |
| CN115134767B (en) * | 2021-03-11 | 2024-02-09 | 上海大唐移动通信设备有限公司 | Method, device and storage medium for improving performance of signaling soft acquisition equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101335709A (en) * | 2008-08-07 | 2008-12-31 | 杭州华三通信技术有限公司 | Method for implementing load sharing among flow analysis servers and shunting equipment |
| CN101640823A (en) * | 2009-09-07 | 2010-02-03 | 杭州华三通信技术有限公司 | Method and equipment for shunting multi-analysis system |
| CN101699802A (en) * | 2009-10-23 | 2010-04-28 | 北京锐安科技有限公司 | Method for branching mass data |
| CN104427534A (en) * | 2013-09-06 | 2015-03-18 | 中国移动通信集团公司 | Detection method and movable detection device of long-term evolution software acqusition |
-
2016
- 2016-04-26 CN CN201610265342.5A patent/CN107318132B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101335709A (en) * | 2008-08-07 | 2008-12-31 | 杭州华三通信技术有限公司 | Method for implementing load sharing among flow analysis servers and shunting equipment |
| CN101640823A (en) * | 2009-09-07 | 2010-02-03 | 杭州华三通信技术有限公司 | Method and equipment for shunting multi-analysis system |
| CN101699802A (en) * | 2009-10-23 | 2010-04-28 | 北京锐安科技有限公司 | Method for branching mass data |
| CN104427534A (en) * | 2013-09-06 | 2015-03-18 | 中国移动通信集团公司 | Detection method and movable detection device of long-term evolution software acqusition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107318132A (en) | 2017-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107318132B (en) | Data distribution method and device in acquisition system | |
| WO2015141337A1 (en) | Reception packet distribution method, queue selector, packet processing device, and recording medium | |
| US20170142177A1 (en) | Method and system for network dispatching | |
| EP3614630B1 (en) | Method for link aggregation and related products | |
| CN105554728A (en) | Cloud SIM card pool system | |
| CN104540234B (en) | A kind of associated task scheduling mechanism synchronously constrained based on CoMP under C RAN frameworks | |
| CN103297282A (en) | Network delay measuring device and method | |
| CN106063201A (en) | Server, control device, management device, communication system, communication method, control method, management method, and program | |
| CN108718258B (en) | Quality detection method and equipment for inter-board link | |
| CN107241701B (en) | data transmission method and device | |
| CN113364682A (en) | Data transmission method and device, storage medium and electronic device | |
| CN105335313B (en) | A kind of transmission method and device of basic data | |
| CN103607731B (en) | A kind of processing method and processing device of measurement report | |
| CN104852756A (en) | Antenna mapping method, apparatus and digital front end | |
| CN107547392A (en) | Data processing method in internet of things | |
| CN106793093B (en) | Service processing method and device | |
| CN111372277B (en) | Data distribution method, device and storage medium | |
| CN109688171B (en) | Cache space scheduling method, device and system | |
| CN108449705B (en) | Method and device for controlling internet access position of wireless broadband user | |
| CN105704173A (en) | Cluster system data distribution method and server | |
| CN103619025B (en) | A kind of network capacity computational methods and equipment | |
| CN103747488B (en) | carrier balancing method, device and system | |
| CN102572753A (en) | Method and system for analyzing signaling of mobile application part | |
| RU2017102506A (en) | BASIC STATION, SERVER, SYSTEM AND METHOD FOR DISTRIBUTING WIRELESS RESOURCES | |
| CN106385385B (en) | Resource allocation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |