CN113438267A

CN113438267A - Method and equipment for analyzing stream data

Info

Publication number: CN113438267A
Application number: CN202010209907.4A
Authority: CN
Inventors: 雷高伟; 杨丹军; 师磊; 高立夫
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-03-23
Filing date: 2020-03-23
Publication date: 2021-09-24
Anticipated expiration: 2040-03-23
Also published as: CN113438267B

Abstract

The embodiment of the application discloses a method and equipment for analyzing stream data, which are used for analyzing the association relationship between servers so as to perform application migration in batches according to the association relationship. The method in the embodiment of the application comprises the following steps: acquiring K pieces of stream data, wherein the stream data comprises information of data sent from a source binary group to a destination binary group, and the binary group comprises an Internet Protocol (IP) address and a port number; then, according to the stream data, M service-side binary groups are identified from the binary groups of the stream data, wherein K is a positive integer, and M is a positive integer; and finally, dividing the M service ends into N service end sets according to the stream data, wherein the service end sets comprise one or more service ends, N is a positive integer, and if the service end sets comprise a plurality of service ends, the plurality of service ends have an association relationship.

Description

Method and equipment for analyzing stream data

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a method and equipment for analyzing stream data.

Background

With the development of cloud technology, more and more clients choose to migrate applications in a local server cluster to a cloud server cluster.

If the applications in all the servers in the local server cluster are migrated to the cloud server cluster together, the workload is large and errors are prone to occur, so before migration, the association relationship among the servers in the local server cluster needs to be analyzed, and then application migration is performed in batches according to the association relationship among the servers. For example, based on the existence of an association relationship between two servers, the two servers may be migrated as the same batch.

Therefore, since the association relationship between servers can be obtained from stream data, a method of analyzing stream data is required.

Disclosure of Invention

The embodiment of the application provides a method and equipment for analyzing streaming data, which are used for analyzing the association relationship between servers so as to perform application migration in batches according to the association relationship.

A first aspect of the embodiments of the present application provides a method for analyzing stream data, which may be applied to a server, and includes:

acquiring K pieces of stream data, wherein the stream data comprises information of data sent from a source binary group to a destination binary group, and the binary group comprises an Internet Protocol (IP) address and a port number;

the stream data may include, in addition to the source doublet and the destination doublet, a start time of transmitting the stream data, an end time of transmitting the stream data, a flag bit, a number of bytes transmitted, and a number of packets transmitted, wherein the flag bit is associated with a communication protocol.

According to the stream data, M service-side binary groups are identified from the binary groups of the stream data, wherein K is a positive integer, and M is a positive integer;

a piece of stream data may not contain a tuple belonging to the server or may contain a tuple belonging to the server.

Dividing the M service ends into N service end sets according to the streaming data, wherein the service end sets comprise one or more service ends, N is a positive integer, and if the service end sets comprise a plurality of service ends, the plurality of service ends have an association relation.

In the embodiment of the application, a binary group belonging to a server side in the K pieces of stream data is identified, so that the corresponding relation between the service and the server can be determined, for example, the number of the services deployed on each server; and then dividing the identified M binary groups into N server side sets, wherein if the server side sets comprise a plurality of server sides, the plurality of server sides have an incidence relation, and the incidence relation can reflect the incidence relation between the servers, so that the application migration can be realized.

Based on the first aspect of the embodiments of the present application, the embodiments of the present application provide a first implementation manner of the first aspect, in which, according to the stream data, identifying M servers from a tuple of the stream data includes one or more of the following:

determining a destination binary group in the first flow data as a server side based on the first flow data containing a flag bit SYN, wherein the first flow data is one of K pieces of flow data;

determining a source binary group in the first flow data as a server based on the flag bit SYN/ACK contained in the first flow data;

determining the source binary group in the first stream data as a server based on the port number of the source binary group in the first stream data being in a preset first range; or

And determining the destination binary group in the first stream data as the server based on the port number of the destination binary group in the first stream data being in a preset first range.

In the embodiment of the application, the duplet belonging to the server side can be identified through the port number and the flag bit.

Based on the first aspect of the embodiment of the present application or the first implementation manner of the first aspect, an embodiment of the present application provides a second implementation manner of the first aspect, where identifying M tuples belonging to a server from the tuples of stream data includes:

acquiring the number of the binary groups communicated with the first binary group according to the K pieces of stream data to serve as the first number;

acquiring the number of the binary groups communicated with the second binary group according to the K pieces of stream data to serve as the second number, wherein the first binary group and the second binary group are binary groups of second stream data, and the second stream data is one of the K pieces of stream data;

and determining the first binary group as a server based on the first number being greater than a preset first threshold and the difference between the first number and the second number being greater than a preset second threshold, wherein the first threshold can be set according to the scale of the server cluster and the deployment condition of the service in the server cluster.

The embodiment of the application provides another feasible scheme for identifying the binary group belonging to the server.

Based on the first aspect of the embodiment of the present application, or the first implementation manner of the first aspect, or the second implementation manner of the first aspect, an embodiment of the present application provides a third implementation manner of the first aspect, where identifying M tuples belonging to a server from the tuples of stream data includes:

acquiring data volume sent from a third tuple to a fourth tuple according to the K pieces of stream data to serve as first data volume, wherein the third tuple and the fourth tuple are both tuples contained in the K pieces of stream data;

acquiring the data volume sent from the fourth tuple to the third tuple according to the K pieces of stream data to serve as a second data volume;

and determining the third binary group as the server side based on the fact that the first data volume is larger than the second data volume.

The embodiment of the application provides another feasible scheme for identifying the two-tuple belonging to the server, and in the scheme, one two-tuple with large data volume sent in two mutually communicated two-tuples is determined as the server.

Based on the first aspect of the embodiment of the present application, or the first implementation manner of the first aspect, or the second implementation manner of the first aspect, or the third implementation manner of the first aspect, an embodiment of the present application provides a fourth implementation manner of the first aspect, where dividing M servers into N server sets according to stream data includes:

obtaining Q first call chains corresponding to M service terminals, wherein the first call chains are used for representing a call relation between at least two service terminals, and A is a positive integer;

dividing M service terminals into X service terminal sets according to Q first call chains, wherein X is a positive integer less than or equal to N;

in one server set of the X server sets, the servers may have a call relationship therebetween.

In the embodiment of the application, the server sets are divided according to the call relation among the servers, so that the servers with the call relation are divided into one server set, and the servers corresponding to the servers with the call relation can be used as the same batch for application migration.

Based on the fourth implementation manner of the first aspect of the embodiment of the present application, an embodiment of the present application provides a fifth implementation manner of the first aspect, where the obtaining Q first call chains corresponding to the M servers includes:

acquiring W second call chains corresponding to the M servers, wherein the second call chains are used for representing a call relation between the two servers, and W is a positive integer;

and then combining a plurality of second call chains with overlapping call relations in the W second call chains into a first call chain.

In the embodiment of the application, first, W second call chains with two layers of call relations are obtained, and then, a plurality of second call chains with overlapping call relations in the W second call chains are combined, so that Q first call chains can be finally obtained.

Based on the fifth implementation manner of the first aspect of the embodiment of the present application, an embodiment of the present application provides the sixth implementation manner of the first aspect, where the obtaining W second call chains corresponding to the M servers includes:

acquiring first characteristic information of a first group of stream data according to the K pieces of stream data, wherein the first group of stream data comprises stream data sent to a first service end by a binary group corresponding to a first IP address; the first characteristic information is used for characterizing the first group of stream data, and the type of the first characteristic information may be various, for example, the first characteristic information may include the number of pieces of stream data and the sending time of each piece of stream data; the first service end is one of the M service ends and corresponds to a second IP address;

acquiring second characteristic information of a second group of stream data according to the K pieces of stream data, wherein the second group of stream data comprises stream data sent to a second server by a binary group corresponding to the first IP address and/or a binary group corresponding to the second IP address; the second characteristic information is used for characterizing the second group of stream data, and the type of the second characteristic information can be various, for example, the second characteristic information includes the number of the stream data and the sending time of each stream data; the second server is one of the M servers;

determining an association relation between the first group of stream data and the second group of stream data according to the first characteristic information and the second characteristic information;

and determining a second call chain according to the first IP address, the first service end and the second service end based on the incidence relation between the first group of stream data and the second group of stream data meeting the preset condition, wherein the second call chain is used for indicating that the first service end calls the second service end.

The embodiment of the application provides a feasible scheme for acquiring W second call chains corresponding to M servers.

Based on the fourth implementation manner of the first aspect of the embodiment of the present application, or the fifth implementation manner of the first aspect, or the sixth implementation manner of the first aspect, an embodiment of the present application provides the seventh implementation manner of the first aspect, in which dividing M servers into X server sets according to Q first call chains includes:

determining the third server as a core server based on the fact that the frequency of the third server appearing in the Q first call chains is greater than a third threshold, wherein the third server is one of the M servers;

the third threshold may be set according to actual needs, for example, may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster;

for one call chain in the Q first call chains, dividing a core service end in the call chain into a core service end set, and dividing other service ends except the core service end into a common service end set, wherein the core service end set comprises one or more core service ends;

and establishing an incidence relation between the core server set and the common server set.

In the embodiment of the application, a core server and a common server can be identified from M servers, the core server in the same first call chain is divided into a core server set, and other servers except the core server are divided into a common server set; in this way, the servers corresponding to the core servers in the core server set can be individually used as one batch for application migration, and the servers corresponding to the common servers in the common server set are used as another batch for application migration, so that the situation that the servers corresponding to all the servers in a plurality of first call chains are used as the same batch for application migration due to the fact that the core servers are simultaneously included in the plurality of first call chains is avoided.

Based on the seventh implementation manner of the first aspect of the embodiments of the present application, an embodiment of the present application provides an eighth implementation manner of the first aspect, where the method further includes:

acquiring the number of IP addresses in a binary group communicated with a fourth server according to the K pieces of stream data to serve as a third number, wherein the fourth server is one of the M servers;

based on that the third number is greater than a fourth threshold, the fourth server is determined as the core server, and the fourth threshold may be set according to actual needs, for example, may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster.

The embodiment of the application provides another feasible scheme for identifying the core server.

Based on the fourth implementation manner of the first aspect of the present application, or the fifth implementation manner of the first aspect, or the sixth implementation manner of the first aspect, or the seventh implementation manner of the first aspect, or the eighth implementation manner of the first aspect, an embodiment of the present application provides a ninth implementation manner of the first aspect, where dividing M servers into N server sets according to stream data includes:

acquiring similarity between a fifth server and a sixth server according to the K pieces of stream data, wherein the fifth server and the sixth server are both one of the M servers;

and based on the fact that the similarity between the fifth server and the sixth server is greater than the preset similarity, dividing the fifth server and the sixth server into a similar server set.

In the embodiment of the application, the server side sets are divided according to the similarity between the server sides, the servers deployed by the same service can be determined according to the obtained similar server side sets, and the servers corresponding to the server sides in the similar server side sets can be used as the same batch for application migration.

Based on the ninth implementation manner of the first aspect of the embodiment of the present application, an embodiment of the present application provides the tenth implementation manner of the first aspect, where the obtaining, according to the K pieces of stream data, a similarity between the fifth server and the sixth server includes:

acquiring H binary groups communicated with a fifth server according to the K pieces of stream data, wherein H is a positive integer;

acquiring feature data of a communication session between the fifth server and the H binary groups according to the K pieces of stream data, wherein the feature data is used for representing the characteristics of the communication session;

obtaining G binary groups communicated with a sixth server according to the K pieces of stream data, wherein G is a positive integer;

acquiring characteristic data of communication conversation between the sixth server and the G binary groups according to the K pieces of stream data;

and calculating the similarity between the fifth server and the sixth server according to the characteristic data of the communication session between the fifth server and the H tuples, the characteristic data of the communication session between the sixth server and the G tuples and a preset distance calculation formula.

The feature data may be selected according to actual needs, and may include one or more of the following items: the number of bytes transmitted in the stream data, the number of packets transmitted in the stream data, the communication duration of the stream data (the difference between the end time of transmitting data and the start time of transmitting data), and the time interval between two adjacent stream data, which can be obtained from the start time of transmitting data in two adjacent stream data.

The embodiment of the application provides a feasible scheme for obtaining the similarity between two service ends.

Based on the ninth implementation manner of the first aspect of the embodiment of the present application or the tenth implementation manner of the first aspect, an embodiment of the present application provides the eleventh implementation manner of the first aspect, in which a fifth server belongs to a first server set and belongs to a similar server set, and the first server set is one of X server sets;

after dividing the fifth server and the sixth server into a set of similar servers, the method further includes:

and merging the first server set and the similar server set to which the fifth server belongs into a server set.

In the embodiment of the present application, the similar server sets are merged with one or more of the X server sets, so as to divide all the servers having an association relationship into one server set as much as possible.

A second aspect of the embodiments of the present application provides an apparatus for analyzing streaming data, which may be applied to a server, and includes:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring K pieces of stream data, the stream data comprises information of data sent from a source binary group to a destination binary group, and the binary group comprises an Internet Protocol (IP) address and a port number;

The identification unit is used for identifying M service-side binary groups from the binary groups of the stream data according to the stream data, wherein K is a positive integer, and M is a positive integer;

The set dividing unit is used for dividing the M service ends into N service end sets according to the stream data, each service end set comprises one or more service ends, wherein N is a positive integer, and if the service end sets comprise a plurality of service ends, the plurality of service ends have an association relation.

Based on the second aspect of the embodiments of the present application, the embodiments of the present application provide a first implementation manner of the second aspect, in which the identifying unit is configured to perform one or more of the following:

Based on the second aspect of the embodiments of the present application, or the first implementation manner of the second aspect, the embodiments of the present application provide a second implementation manner of the second aspect, in which the identifying unit is configured to:

Based on the second aspect of the embodiments of the present application, or the first implementation manner of the second aspect, or the second implementation manner of the second aspect, the embodiments of the present application provide a third implementation manner of the second aspect, in which the identifying unit is configured to:

Based on the second aspect of the embodiments of the present application, or the first implementation manner of the second aspect, or the second implementation manner of the second aspect, or the third implementation manner of the second aspect, the embodiments of the present application provide a fourth implementation manner of the second aspect, in which the set dividing unit is configured to:

Based on the fourth implementation manner of the second aspect of the embodiments of the present application, an embodiment of the present application provides a fifth implementation manner of the second aspect, where the set dividing unit is configured to:

Based on the fifth implementation manner of the second aspect of the embodiments of the present application, an embodiment of the present application provides a sixth implementation manner of the second aspect, and the set dividing unit is configured to:

Based on the fourth implementation manner of the second aspect of the embodiments of the present application, or the fifth implementation manner of the second aspect, or the sixth implementation manner of the second aspect, the embodiments of the present application provide a seventh implementation manner of the second aspect, in which the set dividing unit is configured to:

Based on the seventh implementation manner of the second aspect of the embodiments of the present application, the embodiments of the present application provide an eighth implementation manner of the second aspect, in which the set dividing unit is configured to:

Based on the fourth implementation manner of the second aspect of the embodiments of the present application, or the fifth implementation manner of the second aspect, or the sixth implementation manner of the second aspect, or the seventh implementation manner of the second aspect, or the eighth implementation manner of the second aspect, the embodiments of the present application provide a ninth implementation manner of the second aspect, in which the set dividing unit is configured to:

Based on the ninth implementation manner of the second aspect of the embodiments of the present application, the embodiments of the present application provide a tenth implementation manner of the second aspect, in which the set dividing unit is configured to:

Based on the ninth implementation manner of the second aspect of the embodiment of the present application or the tenth implementation manner of the second aspect, an embodiment of the present application provides an eleventh implementation manner of the second aspect, in which a fifth server belongs to a first server set and belongs to a similar server set, and the first server set is one of X server sets;

and the set dividing unit is used for merging the first service end set and the similar service end set to which the fifth service end belongs into a service end set.

A third aspect of the embodiments of the present application provides an apparatus for analyzing stream data, including: at least one processor and a memory, the memory storing computer-executable instructions executable on the processor, the server performing a method of analyzing streaming data as described in any one of the embodiments of the first aspect above when the computer-executable instructions are executed by the processor.

A fourth aspect of the embodiments of the present application provides a chip or a chip system, where the chip or the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is configured to execute a computer program or instructions to perform the method for analyzing streaming data according to any one of the embodiments of the first aspect.

A fifth aspect of embodiments of the present application provides a computer storage medium for storing computer software instructions for the apparatus for analyzing streaming data, including a program designed for executing the apparatus for analyzing streaming data;

the apparatus for analysing streaming data may be a device for analysing streaming data as described in the second aspect above.

A ninth aspect of embodiments of the present application provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the method for analyzing streaming data as described in any one of the embodiments of the first aspect.

A tenth aspect of the present embodiment provides a system, where the system includes an acquisition module, an association analysis module, a batch planning module, and a migration module;

the acquisition module is used for acquiring the flow data of the first server cluster;

the association analysis module is configured to execute the method for analyzing streaming data according to any one of the embodiments of the first aspect, so as to obtain N server sets, where N is a positive integer;

the batch planning module is configured to determine a migration batch of the server according to the N server collections, for example, a server corresponding to a server in one or more server collections may be used as a migration batch;

the migration module is used for migrating the service deployed on the server in the first server cluster to the second server cluster according to the migration batch in batches.

According to the technical scheme, the embodiment of the application has the following advantages:

acquiring K pieces of stream data, wherein the stream data comprises information of data sent from a source binary group to a destination binary group, and the binary group comprises an Internet Protocol (IP) address and a port number; then, according to the stream data, identifying M service-side tuples from the stream data tuples, so as to determine a correspondence between the service and the servers, for example, the number of services deployed on each server, where K is a positive integer and M is a positive integer; finally, dividing the M service ends into N service end sets according to the streaming data, wherein the service end sets comprise one or more service ends, and N is a positive integer; if the server set comprises a plurality of servers, the plurality of servers have an association relationship, the association relationship can reflect the association relationship between the servers, and the application migration can be performed by using the association relationship.

Drawings

FIG. 1 is a block diagram of a migration system according to an embodiment of the present application;

FIG. 2 is a block diagram illustrating an architecture of a server cluster according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an embodiment of a method for analyzing stream data in an embodiment of the present application;

FIG. 4 is a schematic diagram of an embodiment of identifying a server in an embodiment of the present application;

FIG. 5 is a schematic diagram of another embodiment of identifying a server in the embodiment of the present application;

FIG. 6 is a diagram of an embodiment of dividing a server set in an embodiment of the present application;

FIG. 7 is a schematic diagram of an embodiment of obtaining a first call chain in an embodiment of the present application;

FIG. 8 is a schematic diagram of an embodiment of obtaining a second call chain in an embodiment of the present application;

fig. 9 is a schematic diagram of an embodiment of dividing M servers into X server sets according to Q first call chains in the embodiment of the present application;

FIG. 10 is a schematic diagram of another embodiment of dividing a server set in an embodiment of the present application;

fig. 11 is a schematic diagram of an embodiment of obtaining similarity between servers in the embodiment of the present application;

FIG. 12 is a schematic diagram of an embodiment of an apparatus for analyzing stream data in an embodiment of the present application;

fig. 13 is a schematic structural diagram of an apparatus for analyzing stream data in the embodiment of the present application.

Detailed Description

The embodiment of the application provides a method for analyzing streaming data, which is used for analyzing the association relationship between servers so as to perform application migration in batches according to the association relationship.

The embodiment of the application can be applied to the migration system shown in fig. 1. Referring to fig. 1, an architecture of a migration system in the embodiment of the present application is schematically illustrated. The migration system mainly comprises an original server cluster, a migration device and a target server cluster. The migration device is used for migrating the service deployed on the original server cluster to the target server cluster. The migration device may specifically include an acquisition module, an association relationship analysis module, a batch planning module, a migration module, and the like. The migration device may specifically be constituted by one or more servers. For example, the migration device may be a migration server, and the migration device may also be formed by servers such as a data collection server, a streaming data analysis server, a migration planning server, and a migration server.

The acquisition module is configured to acquire relevant data of the servers in the original server cluster, where the relevant data may be communication data between the servers, for example. And the incidence relation analysis module is used for analyzing incidence relation among the servers in the original server cluster according to the related data. And the batch planning module is used for determining the migration batch of the server according to the association relation. And the migration module is used for migrating the applications deployed in the original server cluster to the target server cluster in batches according to the migration batches.

The method for analyzing the stream data provided by the embodiment of the application can be particularly applied to an incidence relation analysis module in a migration system and used for analyzing the stream data in an original server cluster to obtain the incidence relation among the servers.

Before the above method of analyzing streaming data is introduced, a server cluster is introduced. The server cluster comprises a plurality of servers which can be connected through a switch in a communication mode. Referring to fig. 2, an architecture of a server cluster in the embodiment of the present application is schematically illustrated. The server cluster shown in fig. 2 includes four servers, which are a server a, a server B, a server C, and a server D, and two servers are in communication connection with each other.

One or more applications may be deployed in the server cluster. The application refers to a business application, and may be, for example, a face recognition authentication system, an online payment system, an online friend-making system, and the like.

An application may include one or more services. The following description will be made separately.

An application comprises a service: for example, a Domain Name Server (DNS) resolution service, for example, may constitute an application separately and be deployed on Server D. Server D communicates with server a, server B, and server C, respectively, to perform DNS resolution for them.

An application includes a plurality of services: taking the application of the face recognition authentication system as an example, the application may include three services, namely an authentication service, a face recognition service and a database service. Each service may be deployed on one or more servers in a server cluster.

The following describes a deployment scheme applied to a server cluster by taking the server cluster and the face recognition authentication system shown in fig. 2 as an example. Specifically, the authentication service and the database service may be deployed on server a, and the face recognition service may be deployed on server B and server C at the same time. Based on the deployment of the authentication service, the server a may receive an authentication request from the terminal device, where the authentication request may include a face image. Based on the face recognition service deployed on the server B and the server C, the server a may send the face image to the server B or the server C for face recognition. Taking the server B as an example, after receiving the face image, the server B may identify the corresponding user according to the face image. Based on that the server a is also deployed with a database service, the server B can send the identified user to the server a, so that the server a can query the relevant information of the user according to the identified user.

The following describes the concept of IP addresses, ports, servers and clients.

Each network device corresponds to a fixed internet protocol IP address, which is used to identify the network device. Network devices communicate with the outside world through ports, which are typically identified by port numbers. Since the network devices need to rely on a certain communication Protocol for communication, when two network devices communicate, they usually identify two communicating parties by using a triplet, where the triplet includes an IP address, a port number, and a communication Protocol, where the communication Protocol may be a Transmission Control Protocol (TCP) or a User Datagram Protocol (UDP), and the like.

Taking the server cluster shown in fig. 2 as an example, assume that the IP address of server a is 192.168.1.120, and the port number corresponding to server a is 3345; the IP address of server B is 192.168.1.122, the port number of server B is 4000, and the communication protocol is TCP. Then the communication between server a and server B may be represented as the communication between the triplets 192.168.1.120:3545: TCP and 192.168.1.122:4000: TCP.

In order to identify two communicating parties more vividly, in the process of communication between a server and other network devices (terminal devices or other servers), if the server provides services for the other network devices, a triplet corresponding to the server may be referred to as a server, that is, a party providing services in the communication process, and a triplet corresponding to the other network devices may be referred to as a client, that is, a party requesting services in the communication process.

For example, based on the foregoing, after receiving the authentication request, the server a sends a face image to the server B to request the server B to identify a corresponding user according to the face image, that is, the server B provides a face identification service for the server a; therefore, server A may refer to triplet 192.168.1.120:3545: TCP as a client, and server B may refer to triplet 192.168.1.122:4000: TCP as a server.

The following describes the correspondence between the IP address, the port, the server, and the client, and the server.

Based on the foregoing description, the IP address corresponds to the server one-to-one.

Typically, each time a network device communicates, the network device assigns a port to the communication.

When the network device requests service as a client, the network device randomly allocates ports before requesting service, that is, ports used by the client for each communication are randomly allocated, so that the ports of the client are not fixed. However, when the network device provides a service as a server, the network device allocates a corresponding port according to the provided service, that is, a port used by the server for each communication is determined based on the service, so that the port of the server is fixed.

Taking the server cluster shown in fig. 2 as an example, assume that the first network device is server D, the second network device is server a, and the third network device is server B; the port used by the server D to provide DNS resolution service for the server a is the same as the port used by the server D to provide DNS resolution service for the server B; if server D provides DNS resolution service for server a multiple times, the same port is used for providing DNS resolution service each time.

Assuming that the first network device is a server a and the second network device is a terminal device, the port used by the server a for providing the authentication service for the terminal device is different from the port used for providing the database service.

It can also be seen that the number of servers corresponding to a server is related to the kind of service deployed on the server. If the server is provided with a plurality of services, the server corresponds to a plurality of service terminals; if a service is deployed on a server, the server corresponds to a server. Taking the server cluster shown in fig. 2 as an example, two services are deployed on the server a, so the server a corresponds to two servers and is respectively used for providing an authentication service and a database service, and the server B, the server C and the server D all deploy one service, so the three servers respectively correspond to one server and are correspondingly used for providing a face recognition service, a face recognition service and a DNS resolution service.

If the first network device communicates with other network devices (terminal devices or servers), but the first network device does not provide services (for example, the first network device may request services), the first network device randomly allocates a port for each communication, regardless of the type of the requested services.

Still taking the server cluster shown in fig. 2 as an example, assume that the first network device is server a, the second network device is server B, and the third network device is server D; if the server A and the server B carry out first communication and request the server B to provide face recognition service, the server A randomly allocates a port a for the communication before the communication; if server a communicates with server B a second time and requests server B to provide face recognition service, server a randomly allocates port B for this communication before communication. It can be seen that one server can correspond to a plurality of clients, and the ports of the clients are independent of the type of service requested.

It should be noted that, although one server may correspond to a server or a client, when a port is allocated to a client, the port that is already used as the server is avoided, so that the server and the client do not simultaneously include the same port. That is, there is no triple, belonging to both the client and the server.

Based on the above description, there is an association relationship between servers based on the services deployed in the server cluster; since the service end corresponds to the service, the association relationship between the service ends can reflect the association relationship between the servers. Taking the server cluster shown in fig. 2 as an example, the server B and the server C deploy the same service, that is, the server corresponding to the server B and the server corresponding to the server C have a similar relationship; after receiving an authentication request from the terminal device, the server A sends the face image to the server B to request the server B to perform face recognition, so that a calling relationship exists between a server corresponding to the authentication service on the server A and a server corresponding to the face recognition service on the server B.

To this end, an embodiment of the present application provides a method for analyzing stream data to determine an association relationship between servers: the method comprises the steps of firstly identifying triples belonging to the server side in a server cluster, then dividing the server sides with the association relationship together to obtain a plurality of server side sets, and further obtaining the association relationship among the servers according to the plurality of server side sets. The method provided by the embodiment of the application can be used for analyzing the association relationship among the servers in any server cluster, for example, if application migration is to be performed on the local server cluster, the method provided by the embodiment of the application can be used for analyzing stream data of the local server cluster to obtain the association relationship among the servers in the local server cluster; if application migration is to be performed on the cloud server cluster, the method provided by the embodiment of the application can be used for analyzing stream data of the cloud server cluster so as to obtain an association relationship between servers in the cloud server cluster.

For a port with a determined IP address and port number, the communication protocol is usually fixed, i.e. the IP address and port number can also identify a communication port. Therefore, for convenience of description, the embodiment of the present application uses a binary group composed of an IP address and a port number instead of the triplet group in the foregoing description to describe the method for analyzing stream data provided by the embodiment of the present application.

Those skilled in the art will appreciate that the method is similar for triples, and that replacing the doublet in the following directly with a triplet is sufficient. Therefore, a scheme of replacing the binary group with the triple group is also within the protection scope of the present application.

Specifically, referring to fig. 3, an embodiment of a method for analyzing stream data in an embodiment of the present application is illustrated; the embodiment of the present application provides an embodiment of a method for analyzing stream data, where the method may be applied to a server (such as a stream data analysis server, a service analysis server, an application analysis server, a migration server, and the like), and specifically includes:

step 101, acquiring K pieces of stream data.

Wherein the stream data comprises information of data transmitted from the source duplet to the destination duplet, and the duplet comprises an IP address and a port number. Wherein the streaming data may be obtained from a switch between the servers.

Step 102, according to the stream data, identifying M service-side tuples from the stream data tuples, wherein K is a positive integer and M is a positive integer.

Wherein, a piece of stream data may not contain a tuple belonging to the server, i.e. the source tuple does not belong to the client nor the server; the destination duplet does not belong to either the client or the server.

A piece of streaming data may contain a tuple belonging to the server; namely, the source binary group is a client, and the destination binary group is a server; or the source binary group is the server side, and the destination binary group is the client side.

It is impossible for a piece of data to contain two servers, i.e. at least one of the source doublet and the destination doublet is a server.

It should be noted that there are various methods for identifying a server, and this is not limited in the embodiment of the present application. Several of these methods are described below.

The first method comprises the following steps: the doublet belonging to the server is identified based on a flag in the stream data.

For example, assuming that the communication protocol is, identifying M servers from the binary group of stream data may include: and determining a destination duplet in the first flow data as a server side based on the fact that the first flow data contains a flag bit SYN, wherein the first flow data is one of K pieces of flow data.

Illustratively, identifying M servers from the tuple of streaming data based on the communication protocol being TCP may include: and determining the source binary group in the first flow data as the server side based on the flag bit SYN/ACK contained in the first flow data.

The second method comprises the following steps: the port number based on the doublet identifies the doublet of the server.

It should be noted that, during the communication process, the port number of the service end is fixed, and is usually between 0 and 1023; and the client's port number is randomly assigned among the remaining port numbers, and thus is generally located between 1024 and 65536. The doublet belonging to the server can be identified according to its port number.

Illustratively, identifying M servers from the tuple of stream data may include: and determining the source binary group in the first stream data as the server based on the port number of the source binary group in the first stream data being in a preset first range. For example, the preset first range may be 0 to 1023.

Illustratively, identifying M servers from the tuple of stream data may include: and determining the destination binary group in the first stream data as the server based on the port number of the destination binary group in the first stream data being in a preset first range.

The third method comprises the following steps: such a first tuple is determined as a server: the number of the tuples communicated with the first tuple is greater than a preset first threshold, and the number of the tuples communicated with the first tuple is greater than the number of the tuples communicated with the second tuple, wherein the first tuple and the second tuple are both the tuples of the second stream data, and the second stream data is one of the K pieces of stream data.

Referring to fig. 4, still taking the second stream data as an example, as shown in fig. 4, identifying M tuples belonging to the server from the tuples of the stream data may include:

step 201, acquiring the number of the binary group communicated with the first binary group according to the K pieces of stream data to be used as the first number.

It is understood that the doublet in communication with the first doublet may be the second doublet, and may also be other doublets included in the K pieces of stream data.

Step 202, acquiring the number of the binary group communicated with the second binary group according to the K pieces of stream data to be used as the second number.

Similarly, the binary communicated with the second binary may be the first binary, or may be another binary included in the K pieces of stream data.

Step 203, based on the first number being greater than a preset first threshold and the difference between the first number and the second number being greater than a preset second threshold, the first binary is determined as the server.

The first threshold may be set according to actual needs, for example, may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster; the second threshold may also be set according to actual needs, and may be set to 0, for example.

The fourth method comprises the following steps: such a third tuple is determined as a server: the data volume sent by the third two-tuple to the fourth triplet is larger than the data volume sent by the fourth triplet to the third two-tuple. Please refer to fig. 5.

As shown in fig. 5, identifying M service-side tuples from the stream data tuples may include:

step 301, obtaining a data size sent by the third tuple to the fourth tuple according to the K pieces of stream data, and taking the data size as the first data size, where the third tuple and the fourth tuple are both tuples included in the K pieces of stream data.

It is understood that the stream data contains the number of bytes sent, so the amount of data sent from the third tuple to the fourth tuple can be obtained according to the number of bytes sent in the stream data.

Step 302, acquiring the data volume sent by the fourth tuple to the third tuple according to the K pieces of stream data to use the data volume as the second data volume.

Likewise, the amount of data sent by the fourth tuple to the third tuple can be obtained according to the number of bytes sent in the data.

Step 303, based on the first data amount being greater than the second data amount, determining the third tuple as the server.

It can be understood that, in the process of providing the service to the client by the server, in general, the amount of data sent by the server to the client is greater than the amount of data sent by the client to the server, and therefore, in this embodiment, based on the first amount of data being greater than the second amount of data, the third two-tuple is determined as the server.

In the embodiment of the present application, one or more of the above four methods may be adopted to identify the binary group belonging to the server.

For the sake of understanding, the following description specifically describes the above four methods with a specific example.

In this example, it is assumed that 12 pieces of stream data shown in table 1 are acquired, and the binary group in table 1 is specifically shown in table 2.

TABLE 1

TABLE 2

Binary group	IP address	Port number
			Binary 1	IP address 10	30000
Binary 2	IP address 20	443
			Binary 3	IP address 30	40000
Binary 4	IP address 10	50000

Based on the stream data in table 1 and the binary group in table 2, the server is identified by a first method:

the doublet 2 can be determined as the server according to the flag bits SYN in the stream data 1, the stream data 2 and the stream data 9.

The doublet 2 may also be determined as the server according to the flag bits in the stream data 3, the stream data 4 and the stream data 10 being SYN/ACK.

Based on the stream data in table 1, the server is identified by a second method:

the port number 443 of the doublet 2 corresponds to, and since 443 is between 0 and 1023, the doublet 2 can be determined as the server.

Based on the stream data in table 1, the server is identified by a third method:

taking the stream data 1 containing the doublet 1 and the doublet 2 as an example, the doublet communicating with the doublet 1 includes only the doublet 2, and the doublet communicating with the doublet 2 includes the doublet 3 and the doublet 4 in addition to the doublet 1; therefore, the number of the doublet communicating with the doublet 1 is 1, the number of the doublet communicating with the doublet 2 is 3, and the doublet 2 can be determined as the server assuming that the first threshold is 1 and the second threshold is 0.

Based on the stream data in table 1, the server is identified by a fourth method:

from the stream data shown in table 1, the data amount of the communication session (session) shown in table 3 below can be obtained.

TABLE 3

Taking dialog 1 as an example, two communication parties in the dialog 1 are a binary 1 and a binary 2; in the stream data 1, the number of bytes transmitted from the tuple 1 to the tuple 2 is 100, and in the stream data 5, the number of bytes transmitted from the tuple 1 to the tuple 2 is also 100, so that the total number of bytes transmitted from the tuple 1 to the second root 2 is 200.

In stream data 4, the number of bytes transmitted by doublet 2 to doublet 1 is 200, and in stream data 7, the number of bytes transmitted by doublet 2 to doublet 1 is 300, so the total number of bytes transmitted by doublet 2 to the second ancestor 1 is 500. And determining the binary group 2 as the server side based on the fact that the total number of bytes sent by the binary group 2 to the second binary group 1 is greater than the total number of bytes sent by the binary group 1 to the second binary group 2.

And 103, dividing the M service ends into N service end sets according to the stream data.

The server set comprises one or more servers, and N is a positive integer.

If the server set comprises a plurality of servers, the plurality of servers have an incidence relation; the association relationship may be various, for example, may be a similar relationship, and may also be a calling relationship.

Specifically, taking the server a, the server B, and the server C shown in fig. 2 as an example, the face recognition service is deployed on the server B and the server C, so that a server corresponding to the face recognition service on the server B is similar to a server corresponding to the face recognition service on the server C, and therefore the two servers can be divided into a server set.

For another example, in a face recognition authentication system, after receiving an authentication request from a terminal device, a server a sends a face image to a server B or a server C to request the server B or the server C to perform face recognition, so that a call relationship exists between a server corresponding to an authentication service on the server a and a server corresponding to a face recognition service on the server B, and thus the server corresponding to the authentication service on the server a and the server corresponding to the face recognition service on the server B can be divided into a server set.

It should be noted that there are various methods for dividing M servers into N server sets, which are not limited in the embodiment of the present application.

Specifically, servers corresponding to servers in the same server set can be migrated as one batch, and due to the fact that the servers in the same server set have an association relationship, the association relationship also exists in the servers in the same batch, so that the situation that multiple servers without the association relationship are located in the same batch is avoided, and the situation that migration work is difficult to implement due to the fact that excessive data are migrated in the same batch is avoided.

Moreover, based on the correspondence between the service side and the service type, the service side having the association relationship also deploys the service having the association relationship, for example, the service side in one or more service side sets deploys the service of the same application. And after the server corresponding to the server in the server set is used as a batch for application migration, the application can be put into use quickly.

Based on the foregoing description, there are various methods for dividing M servers into N server sets, and it can be known from the relevant description in fig. 2 that there may be a call relationship between different services, and correspondingly, there may also be a call relationship between servers providing services; the same service can be deployed on a plurality of servers, and correspondingly, similar relations exist among different service terminals providing the same service; based on the calling relationship and the similar relationship, two methods for dividing the server set are introduced below.

The first method is to divide the server set according to the call relationship between the servers, and will be described with reference to fig. 6 to 9. The second method is to divide the server set according to the similarity relationship between the servers, and will be described with reference to fig. 10 to 11.

The first method is described below with reference to fig. 6 to 9. Specifically, please refer to fig. 6, which is a schematic diagram illustrating an embodiment of dividing a server set according to an embodiment of the present application. As shown in fig. 6, based on the foregoing embodiments, another embodiment of the method for analyzing streaming data is provided in the embodiments of the present application, where dividing M servers into N server sets according to the streaming data includes:

step 401, obtaining Q first call chains corresponding to M servers.

The first call chain is used for representing a call relation between at least two service ends, and A is a positive integer.

For example, assuming that the first call chain represents that server a calls server B, the first call chain may be: a server a-a server B, where a form of the first call chain is not specifically limited in this embodiment of the application.

Assuming that the first call chain indicates that the server a calls the server B and the server B calls the server C, the first call chain may be the server a-the server B-the server C.

It should be noted that there are various methods for obtaining Q first call chains corresponding to M servers, which are not limited in this embodiment of the present application. One of which will be described later in connection with fig. 7.

Step 402, dividing the M service terminals into X service terminal sets according to the Q first call chains, where X is a positive integer less than or equal to N.

Since the first call chain is used for representing the call relationship between the servers, the embodiments of the present application can divide M servers into X server sets according to the call relationship between the servers.

It should be noted that there are various methods for dividing M servers into X server sets according to Q first call chains.

For example, a method for dividing M servers into X server sets according to Q first call chains includes: the server in each first call chain can be divided into a server set, and finally Q server sets can be obtained; because the calling relationship exists between the service ends in the service end set, the calling relationship exists between the servers corresponding to the service ends in the service end set. If the servers corresponding to the servers in one server set are migrated as the same batch, the services with the calling relationship can be migrated to the target server cluster at one time, so that the servers can be put into use as soon as possible.

One method for obtaining Q first call chains corresponding to M servers is described below with reference to fig. 7. Specifically, as shown in fig. 7, obtaining Q first call chains corresponding to M servers includes:

step 501, obtaining W second call chains corresponding to M servers, where the second call chains are used to represent a call relationship between two servers, and W is a positive integer.

It will be appreciated that in this embodiment, a second call chain representing a call relationship between two servers is first obtained. There are various methods for obtaining W second call chains corresponding to M servers, which are not limited in this embodiment of the present application. A method for obtaining W second call chains corresponding to M servers will be described later with reference to fig. 8.

For ease of understanding, please refer to table 4, where table 4 shows 5 second call chains, where the representation of the second call chains is not limited to the one in table 4.

TABLE 4

Step 502, combining multiple second call chains with overlapping call relations in the W second call chains into one first call chain.

It is assumed here that a second call chain is: a first IP address-a first server-a second server; the second call chain may represent a call relationship between the first IP address and the first service end, or may represent a call relationship between the first service end and the second service end, that is, a two-layer call relationship.

Illustratively, assume that there is another second call chain: a second IP address-a second server-an eighth server; the second call chain represents a call relationship between the second IP address and the second server, and may also represent a call relationship between the second server and the eighth server.

Assuming that the IP address of the first server is the second IP address, the call relationship between the first server and the second server overlaps with the call relationship between the second IP address and the second server, and at this time, the second call chain (the first IP address-the first server-the second server) and the second call chain (the second IP address-the second server-the eighth server) may be combined to remove the overlapping call relationship, so as to obtain a first call chain of the first IP address-the first server-the second server-the eighth server.

Illustratively, assume that there is also a second call chain: a first IP address, a first server and an eighth server; the second call list represents a call relationship between the first IP address and the first service end, and may also represent a call relationship between the first service end and the eighth service end.

Since the second call chains each include a call relationship between the first IP address and the first service end, the second call chain (the first IP address-the first service end-the second service end) and the second call chain (the first IP address-the first service end-the eighth service end) may be combined to remove the repeated call relationship, thereby obtaining a first call chain of the first IP address-the first service end- (the second service end or the eighth service end).

Taking 5 second call chains shown in table 4 as an example, combining multiple second call chains with overlapping call relations among the 5 second call chains, 3 first call chains are finally obtained, specifically please refer to table 5.

TABLE 5

As can be seen from the above description, in this embodiment, first, W second call chains having two layers of call relationships are obtained, and then, multiple second call chains having overlapping call relationships in the W second call chains are combined, so that Q first call chains can be finally obtained.

An embodiment of obtaining W second call chains corresponding to M servers is described below with reference to fig. 8. In this embodiment, a process of determining a second call chain is described by taking a first service end and a second service end as an example. The first server is one of the M servers, and the second server is one of the M servers.

It can be understood that, before the second call chain is established, an association relationship between the first service end and the second service end is first determined, and if the association relationship satisfies the call relationship, the second call chain including the first service end and the second service end may be established.

As can be seen from the foregoing description, in a piece of stream data, at most one service end is included, so to determine the association relationship between the first service end and the second service end, the association relationship between the stream data including the first service end and the stream data of the second service end may be analyzed.

Specifically, as shown in fig. 8, acquiring W second call chains corresponding to M servers includes:

step 601, acquiring first characteristic information of a first group of stream data according to the K pieces of stream data.

The first group of stream data comprises a group of stream data between the first service end and the second tuple corresponding to the first IP address. For any piece of stream data in the first set of stream data, the binary group corresponding to the first IP address belongs to the client and is a source binary group, and the first server is a destination binary group. For convenience and description, an IP address corresponding to the first service terminal is referred to as a second IP address.

It should be noted that the first group of stream data indicates that the binary group corresponding to the first IP address accesses the first service end (for example, requests the first service end to provide service). Since the port number of the binary group corresponding to the first IP address is randomly allocated as the port number of the client IP address when accessing the first server each time, the first IP address may correspond to one binary group or a plurality of binary groups.

Taking the stream data shown in table 1 as an example, assuming that the first IP address is IP address 10, it can be known from table 2 that the two-tuple corresponding to the first IP address is two-tuple 1 and two-tuple 4. Assuming that the doublet 2 is the first service end, the first set of stream data includes stream data 1, stream data 2, stream data 9, and stream data 11. The number of pieces of stream data in the first set of stream data is 4.

The first characteristic information is used to characterize the first group of stream data, and the kind of the first characteristic information may be various, for example, the first characteristic information may include the number of pieces of stream data and the transmission time of each piece of stream data.

And step 602, acquiring second characteristic information of a second group of stream data according to the K pieces of stream data.

The second group of flow data includes flow data between the second service end and the second tuple corresponding to the first IP address, or flow data between the second service end and the second tuple corresponding to the second IP address. For any stream data in the second group of stream data, the second server is a destination binary group, the binary group corresponding to the first IP address is a source binary group or the binary group corresponding to the second IP address is a source binary group, and both the binary group corresponding to the first IP address and the binary group corresponding to the second IP address are clients.

Similarly, the second group of stream data indicates that the binary group corresponding to the first IP address accesses the second server, and the binary IP address corresponding to the second IP address accesses the second server, where the second IP address may correspond to one binary group or a plurality of binary groups.

Similarly, the second characteristic information is used to characterize the second group of stream data, and the type of the second characteristic information may be various, for example, the second characteristic information includes the number of pieces of stream data and the transmission time of each piece of stream data.

Step 603, determining the association relationship between the first group of stream data and the second group of stream data according to the first characteristic information and the second characteristic information.

It can be understood that the first set of stream data indicates that the client corresponding to the first IP address accesses the first server, which accordingly means that the first server provides the first service for the client corresponding to the first IP address; the second group of stream data indicates that the client corresponding to the second IP address or the client corresponding to the first IP address accesses the second server, which correspondingly means that the second server provides the second service for the client corresponding to the first IP address or the client corresponding to the second IP address; and the IP address of the first service end is the second IP address, so the association between the first group of stream data and the second group of stream data can reflect the association between the first service end and the second service end, that is, the association between the first service and the second service.

There are various methods for determining the association relationship according to the first characteristic information and the second characteristic information.

As one implementation manner, the number of pieces of stream data in the first feature information is used as the number of times of accessing the first server by the binary group corresponding to the first IP address, which is referred to as a first number of times; according to statistics of the sending time of the stream data in the first characteristic information and the sending time of the stream data in the second characteristic information, in a preset time period after the binary group corresponding to the first IP address accesses the first server, the number of times that the binary group corresponding to the first IP address and the binary group corresponding to the second IP address access the second server is called as a second number of times; the preset time period may be set according to actual needs, and may be set to 5 minutes, for example.

Then, the ratio of the second times to the first times is used as a first ratio, the size of the first ratio is used as a standard for measuring the strength of the association between the first group of stream data and the second group of stream data, and the larger the product is, the stronger the association is.

The above implementation is described below with a specific example. As shown in table 6, 8 sets of stream data are contained in this example.

TABLE 6

In the stream data shown in table 6, it is assumed that the first group of stream data is a group of stream data with sequence number 1, that is, the IP address corresponding to the client is IP address 11, the server is binary 33, and the IP address corresponding to the server is IP address 33; the second set of stream data may be a set of stream data with sequence number 2, i.e. the IP address corresponding to the client is IP address 33, the server is binary 77, and the IP address corresponding to the server is IP address 77.

The data shown in table 7 below can be obtained by the above-described method of calculating the first frequency and the second frequency based on table 6.

TABLE 7

As can be seen from table 7, there are five pairs of combinations, each pair comprising a first set of stream data and a second set of stream data. Assuming that the first group of stream data is an IP address 11/binary 33, where the binary corresponding to the IP address 11 is a client, and the binary 33 represents a server; the second group is IP address 33/couple 77, where the couple corresponding to IP address 33 is client and couple 77 represents server.

As can be seen from table 7, the number of times that the doublet corresponding to the IP address 11 accesses the doublet 33 belonging to the server is 300, and the number of times that the doublet corresponding to the IP address 33 accesses the doublet 77 belonging to the server is 170 in the preset time period after the doublet corresponding to the IP address 11 accesses the doublet 33 belonging to the server, so that the first ratio is 170/300-0.56.

In order to more accurately determine the association relationship between the first group of stream data and the second group of stream data, as another implementation manner, on the basis of the previous implementation manner, the number of pieces of stream data in the second group of stream data is referred to as the number of times that the second server is accessed by taking the two-tuple corresponding to the first IP address and the two-tuple corresponding to the second IP address in the whole statistical time period, which is referred to as a third number of times.

And taking the ratio of the second times to the third times as a second ratio, and calculating the product of the first ratio and the second ratio. Finally, the size of the product is used as a standard for measuring the strength of the association between the first group of stream data and the second group of stream data, and the larger the product is, the stronger the association is.

Still referring to table 7, assume that the first set of stream data is IP address 11/tuple 33, where the tuple corresponding to the IP address 11 is the source tuple and belongs to the client, and the tuple 33 represents the server; the second group is IP address 33/couple 77, where the couple corresponding to the IP address 33 is the source couple and belongs to the client, and the couple 77 represents the server.

As can be seen from table 7, assuming that the number of times that the binary corresponding to the IP address 11 accesses the binary 33 belonging to the server is 200 times in all the time periods, the second ratio is 170/200 ═ 0.85, and the product of the first ratio and the second ratio is 0.85 × 0.56 ═ 0.476.

Step 604, based on that the association relationship between the first group of stream data and the second group of stream data satisfies the preset condition, determining a second call chain according to the first IP address, the first server and the second server, where the second call chain is used to indicate that the first server calls the second server.

It can be understood that the preset condition may be set according to actual needs, so that when the association relationship between the first group of stream data and the second group of stream data satisfies the preset condition, the first server and the second server can satisfy the call relationship. In this way, a second call chain from the first IP address to the first server and then to the second server can be generated.

Similar to the first call chain, the form of the second call chain is not particularly limited in the embodiments of the present application. For example, the second call chain may be a first IP address-a first server-a second server.

Assuming that the magnitude of the first ratio is used as a standard for measuring the strength of the association relationship between the first group of stream data and the second group of stream data, the preset condition may be that the first ratio is greater than a ratio threshold; assuming that the size of the product is used as a criterion for measuring the strength of the association between the first set of stream data and the second set of stream data, the preset condition may be that the product is greater than the product threshold.

Taking the data shown in table 7 as an example, assume that the magnitude of the product is used as a criterion for measuring the strength of the association between the first set of stream data and the second set of stream data, and the first threshold is 0.25; then, if the product is greater than 0.25, a second call chain may be determined according to the first IP address, the first server, and the second server. From the data shown in table 7, 5 second call chains as shown in table 4 can be finally generated.

Another method for dividing M servers into X server sets according to Q first call chains is described below with reference to fig. 9. The method is based on the former method, and further divides a server set composed of the servers in a first call chain.

Based on the foregoing embodiments, in another embodiment of the method for analyzing streaming data provided in this embodiment of the present application, dividing M servers into X server sets according to Q first call chains includes:

step 701, based on that the number of times that the third server appears in the Q first call chains is greater than a third threshold, determining the third server as a core server, where the third server is one of the M servers.

The third threshold may be set according to actual needs, for example, may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster.

For example, assuming that M servers correspond to 8 first call chains, the third threshold is 6, and the third server is included in each first call chain, that is, the number of times that the third server appears in Q first call chains is 8, the third server may be determined as the core server.

Taking 3 first call chains shown in table 5 as an example, the third threshold value of the IP address is 1; it can be seen that the number of times that the server 3 appears in the 3 first call chains is 2, and the number of times that the server 7 appears in the 3 first call chains is 1, so that the server 3 and the server 7 can be determined as core servers.

It is understood that the core server is used to provide services for a plurality of clients, for example, a server corresponding to the DNS service may provide domain name resolution services for many clients.

Step 702, for one of the Q first call chains, dividing the core server therein into a core server set, and dividing the other servers except the core server into a common server set, where the core server set includes one or more core servers.

It can be understood that, assuming that a first call chain includes 8 service terminals, of which 2 are core service terminals, the 2 core service terminals may be divided into a core service terminal set, and then the other 6 service terminals except the 2 core service terminals are divided into a common service terminal set. Optionally, the 2 core servers may also be divided into two core server sets, where each core server set includes one of the core servers.

Taking 3 first call chains shown in table 5 as an example, assuming that the server 3 and the server 7 are core servers, the finally obtained core server set may be a server set including both the server 3 and the server 7, and the common server set includes two sets, one is a server set including the server 4, and the other is a server set including both the server 5 and the server 6.

Step 703, establishing an association relationship between the core server set and the common server set.

Finally, the core server set and the common server set are associated in the embodiment of the application.

When the core server is included in multiple call chains, if the core server is not divided separately, the migration design may cause a situation in which the servers corresponding to all the servers in the multiple call chains are migrated as the same batch of applications, which may make the migration difficult to implement. In the embodiment of the application, a core server and a common server can be identified from M servers, the core server in the same first call chain is divided into a core server set, other servers except the core server are divided into a common server set, and an association relationship between the core server set and the common server set is established. Therefore, the calling relation among the service terminals in the same calling chain is reserved, the relation among the service terminals can be clearer, and the design of application migration is facilitated.

For example, the application migration may be performed by taking a server corresponding to a core server in the core server set as a batch alone, and the application migration may be performed by taking a server corresponding to a common server in the common server set as another batch. Moreover, the application migration may be performed on the server corresponding to the core server in the core server set, and then the migration is performed on the server corresponding to the common server in the common server set. Therefore, after the migration of the server corresponding to the core server is completed, the core server can be utilized to provide the core service.

In addition, when the number of the core servers in the first call chain is large, the embodiment of the application can also avoid the situation that all the servers in the first call chain are divided into one server set to cause the number of the servers in the server set to be too large.

In the above embodiment, the core server is identified by the number of times that the server appears in the first call chain, and it should be noted that, in addition, the core server in the M servers may be identified by other methods.

Illustratively, based on the foregoing embodiments, in another embodiment of the method for analyzing streaming data provided in the embodiments of the present application, the method further includes:

and determining the fourth server as the core server based on the third number being greater than the fourth threshold.

It can be understood that the greater the number of IP addresses communicated with the fourth server, the more basic and extensive the services provided by the fourth server are, so the core server can be identified according to the number of IP addresses communicated with the fourth server.

The fourth threshold may be set according to actual needs, for example, may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster.

A second method (dividing the server set according to the similarity relationship between the servers) is described below with reference to fig. 10 to 11.

Based on the foregoing embodiments, in another embodiment of the method for analyzing streaming data provided in the embodiment of the present application, as shown in fig. 10, dividing M servers into N server sets according to an association relationship between the servers further includes:

step 801, obtaining similarity between a fifth server and a sixth server according to the K pieces of flow data, where the fifth server and the sixth server are both servers in the M servers.

It should be noted that there are various methods for obtaining the similarity between the fifth server and the sixth server, which is not limited in this embodiment of the present application, and one of the methods will be described later with reference to fig. 11.

Step 802, based on that the similarity between the fifth server and the sixth server is greater than the preset similarity, the fifth server and the sixth server are divided into a similar server set.

It can be understood that similar relationships exist among a plurality of servers in a set of similar servers; the preset similarity may also be set according to actual needs, for example, the preset similarity may be set according to the scale of the server cluster and the deployment situation of the service in the server cluster.

Therefore, in the embodiment of the application, the servers deployed by the same service can be determined according to the similar server set, and the servers corresponding to the servers in the similar server set can also be used as the same batch for application migration, even if multiple servers deployed by the same service can be used as the same batch for application migration.

Referring to fig. 11, an illustration of an embodiment of obtaining similarity between servers in the embodiment of the present application is shown. As shown in fig. 11, based on the foregoing embodiment, in another embodiment of the method for analyzing streaming data provided in this embodiment of the present application, the obtaining a similarity between the fifth server and the sixth server according to the K pieces of streaming data includes:

and step 901, acquiring H binary groups communicated with a fifth server according to the K pieces of stream data, wherein H is a positive integer.

It can be understood that H tuples communicated with the fifth server can be determined according to the source doublet and the destination doublet contained in the K pieces of stream data.

And step 902, acquiring feature data of the communication session between the fifth server and the H tuples according to the K pieces of stream data, wherein the feature data is used for representing characteristics of the communication session.

It should be noted that the communication session is determined by both communication parties, specifically, assuming that the fifth tuple is one of H tuples, the communication session in which the fifth tuple is a source tuple and the fifth server is a destination tuple, and the communication session in which the fifth tuple is a destination tuple and the fifth server is a source tuple belong to the same communication session, so that one communication session may include one or more pieces of streaming data.

Based on the relevant descriptions in table 1 and table 2, it can be known that the binary group 2 is finally determined as the server; for ease of understanding, it is assumed here that the fifth server is doublet 2 in table 1, and from table 1 and table 2, the characteristic data of the communication session in table 8 containing doublet 2 can be obtained.

TABLE 8

Step 903, obtaining G binary groups communicated with a sixth server according to the K pieces of stream data, wherein G is a positive integer;

similarly, G tuples to communicate with the sixth server can be determined according to the source doublet and the destination doublet included in the K pieces of stream data.

It should be noted that, there is no sequence when step 901 and step 903 are executed.

And 904, acquiring the characteristic data of the communication session between the sixth server and the G binary groups according to the K pieces of stream data.

It should be noted that, the step 904 may be understood with reference to the related description of the step 902, and the description of the embodiment of the present application is not repeated herein.

Step 905, calculating the similarity between the fifth server and the sixth server according to the feature data of the communication session between the fifth server and the H tuples, the feature data of the communication session between the sixth server and the G tuples, and a preset distance calculation formula.

The distance calculation formula may be a campbera distance calculation formula, an Euclidean distance calculation formula, or the like.

It should be noted that there are various methods for calculating similarity according to the feature data and the distance calculation formula, and this is not limited in the embodiments of the present application.

Illustratively, first, a stream feature is extracted according to feature data of a communication session between the fifth server and H tuples, and the stream feature may include a mean, a standard deviation, a sum, a first order difference, or the like. For example, assuming that the feature data is the number of bytes sent in the stream data, the fifth server corresponds to H communication sessions with H tuples, and any communication session may contain a plurality of stream data. If one of the H communication sessions contains 8 pieces of stream data and the feature data includes the number of bytes sent in the stream data, extracting the stream feature from the feature data of the communication session may include: the average value of the number of bytes sent in 8 pieces of stream data is obtained.

Referring to table 9, as an example, the flow characteristics extracted from the characteristic data of H communication sessions corresponding to the fifth server are shown. Where H is 3 and the stream characteristics include mean, sum and standard deviation.

TABLE 9

Then, stream characteristics are extracted according to the characteristic data of the communication session between the sixth server and the G tuples, and the stream characteristics may also include a mean value, a standard deviation, a sum value, a first order difference, and the like.

And finally, calculating the similarity between the fifth server and the sixth server by adopting a distance calculation formula according to the flow characteristics corresponding to the communication session between the fifth server and the H binary groups and the flow characteristics corresponding to the communication session between the sixth server and the G binary groups.

Based on the foregoing description, the server sets are divided according to the call relationship between the servers, and finally, the M servers can be divided into X server sets; dividing a server set according to the similarity relation among the servers, and finally obtaining a similar server set; that is, the N server sets may simultaneously include the similar server set and the X server sets.

It should be noted that, in general, the service terminals in the similar service terminal set may provide services of the same type, and therefore, the similar service terminal set may be merged with one or more service terminal sets of X service terminal sets, so as to divide all the service terminals having an association relationship into one service terminal set as much as possible.

Specifically, in another embodiment of the method for analyzing streaming data provided in the embodiment of the present application, if the fifth server belongs to the first server set and belongs to a similar server set, the first server set is one of X server sets; the method further comprises: and merging the first server set and the similar server set to which the fifth server belongs into a server set.

Specifically, for example, the application of the server cluster and the face recognition authentication system shown in fig. 2 is assumed that the server providing the authentication service is a server a1, the servers providing the face recognition service are a server B1 and a server C1, the server providing the database service is a server a2, and the server providing the DNS service is a server D1; based on the calling relationship among the authentication service, the face recognition service, the database service and the DNS service, the server A1, the server B, the server A2 and the server D1 belong to the same calling chain, so the server A1, the server B, the server A2 and the server D1 can be divided into one of X server sets; based on the fact that the face recognition service is deployed on the server B and the server C at the same time, the server B1 and the server C1 can be divided into similar server sets.

In the embodiment of the present application, the similar server set to which the server B1 and the server C1 belong is finally merged with the server sets to which the server a1, the server B, the server a2, and the server D1 belong.

Based on the foregoing description, the X server sets may include a core server set and a common server set; in this case, as an implementation manner, the merging the first server set and the similar server set to which the fifth server belongs into one server set may specifically include:

based on the fifth server being a common server, merging a common server set where the fifth server is located and a similar server set where the fifth server is located into a server set; and/or

And combining the core server set where the fifth server is located and the similar server set where the fifth server is located into a server set based on the fifth server being a core server.

Specifically, taking the application of the server cluster and the face recognition authentication system shown in fig. 2 as an example, based on the fact that there is a call relationship among the authentication service, the face recognition service, the database service, and the DNS service, the server a1, the server B, the server a2, and the server D1 belong to the same call chain; since the DNS service is a core service, the server D1 corresponding to the DNS service is a core server, and based on the foregoing embodiment, the server D1 in the call chain can be divided into a core server set, and the server a1, the server B, and the server a2 can be divided into a common server set.

Based on the fact that the face recognition service is deployed on the server B and the server C at the same time, the server B1 and the server C1 can be divided into similar server sets.

In the embodiment of the present application, the similar server sets to which the server B1 and the server C1 belong may be merged with the server sets to which the server a1, the server B, and the server a2 belong.

Finally, a core server set including server D1 is obtained, and a server set including server a1, server B, server C, and server a2 is obtained.

As can be seen from the foregoing description, application migration may be performed according to a server set. Specifically, the server a, the server B, and the server C corresponding to the server a1, the server B, the server C, and the server a2 may be used as the same batch for application migration; similarly, the application migration may be performed by taking the server D corresponding to the server D1 as a batch.

Referring to fig. 12, an embodiment of the present application provides an apparatus for analyzing stream data, including:

an obtaining unit 100, configured to obtain K pieces of stream data, where the stream data includes information of data sent from a source binary group to a destination binary group, and the binary group includes an internet protocol IP address and a port number;

An identifying unit 200, configured to identify M tuples belonging to a server from the tuples of the stream data according to the stream data, where K is a positive integer and M is a positive integer;

The set dividing unit 300 is configured to divide the M servers into N server sets according to the stream data, where a server set includes one or more servers, where N is a positive integer, and if the server set includes multiple servers, the multiple servers have an association relationship.

In another embodiment of the apparatus for analyzing streaming data provided by the embodiment of the present application, the identification unit 200 is configured to perform one or more of the following:

In another embodiment of the apparatus for analyzing stream data provided in the embodiment of the present application, the identification unit 200 is configured to:

In another embodiment of the apparatus for analyzing stream data provided in the embodiment of the present application, the set partitioning unit 300 is configured to:

In another embodiment of the apparatus for analyzing stream data provided in the embodiment of the present application, the fifth server belongs to the first server set and belongs to a similar server set, and the first server set is one of X server sets;

a set dividing unit 300, configured to merge the first server set and the similar server set to which the fifth server belongs into a server set.

Referring to fig. 13, an embodiment of an apparatus for analyzing streaming data according to an embodiment of the present application may include one or more processors 400, a memory 500, and a communication interface 600.

The memory 500 may be transient or persistent. Still further, the processor 400 may be configured to communicate with the memory 500 to execute a series of instruction operations in the memory 500 on a server.

In this embodiment, the processor 400 may perform the operations performed by the apparatus for analyzing stream data in the embodiment shown in fig. 12, which are not described herein again.

In this embodiment, the specific functional block division in the processor 400 may be similar to the functional block division described in fig. 12, and is not described herein again.

An embodiment of the present application further provides a chip or a chip system, where the chip or the chip system includes at least one processor and a communication interface, the communication interface is interconnected with the at least one processor through a line, and the at least one processor is configured to run a computer program or an instruction to execute operations executed by the apparatus for analyzing streaming data in the embodiment shown in fig. 12, which is not described herein again in detail.

The communication interface in the chip may be an input/output interface, a pin, a circuit, or the like.

The embodiments of the present application further provide a first implementation manner of a chip or a chip system, where the chip or the chip system described above in the present application further includes at least one memory, and the at least one memory stores instructions therein. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or may be a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

Embodiments of the present application also provide a computer storage medium for storing computer software instructions for the above apparatus for generating a task guide, which includes a program designed to execute the apparatus for analyzing streaming data.

The apparatus for analyzing streaming data may be the apparatus for analyzing streaming data as described in the foregoing fig. 12.

Embodiments of the present application further provide a computer program product, where the computer program product includes computer software instructions, and the computer software instructions can be loaded by a processor to implement the flow of the method in fig. 3 to 11.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Claims

1. A method of analyzing streaming data, comprising:

according to the stream data, M service-side tuples are identified from the stream data tuples, wherein K is a positive integer, and M is a positive integer;

dividing the M service ends into N service end sets according to the streaming data, wherein the service end sets comprise one or more service ends, N is a positive integer, and if the service end sets comprise a plurality of service ends, the service ends have an association relationship.

2. The method of claim 1, wherein the identifying M servers from the tuple of flow data according to the flow data comprises one or more of:

determining a destination binary group in first stream data as a server based on that the first stream data contains a flag bit SYN, wherein the first stream data is one of the K pieces of stream data;

determining a source binary group in first flow data as a server based on the first flow data containing a flag bit SYN/ACK;

determining a source binary group in first stream data as a server based on a port number of the source binary group in the first stream data being in a preset first range; or

And determining the destination binary group in the first stream data as a server based on the port number of the destination binary group in the first stream data being in a preset first range.

3. The method according to claim 1 or 2, wherein the identifying M service-side tuples from the stream data tuples comprises:

and determining the first binary group as the server based on the fact that the first number is larger than a preset first threshold value and the difference value between the first number and the second number is larger than a preset second threshold value.

4. The method according to any one of claims 1 to 3, wherein the identifying M service-side tuples from the stream data tuples comprises:

acquiring data volume sent from a third tuple to a fourth tuple according to the K pieces of stream data to use the data volume as a first data volume, wherein the third tuple and the fourth tuple are both tuples contained in the K pieces of stream data;

acquiring the data volume sent by the fourth tuple to the third tuple according to the K pieces of stream data to serve as a second data volume;

and determining the third tuple as a server based on the first data volume being larger than the second data volume.

5. The method according to any one of claims 1 to 4, wherein the dividing the M servers into N sets of servers according to the streaming data comprises:

obtaining Q first call chains corresponding to the M service terminals, wherein the first call chains are used for representing a call relation between at least two service terminals, and A is a positive integer;

and dividing the M service ends into X service end sets according to the Q first call chains, wherein X is a positive integer less than or equal to N.

6. The method according to claim 5, wherein the obtaining of Q first call chains corresponding to the M servers comprises:

and combining a plurality of second call chains with overlapping call relations in the W second call chains into a first call chain.

7. The method according to claim 6, wherein the obtaining W second call chains corresponding to the M servers comprises:

acquiring first characteristic information of a first group of stream data according to the K pieces of stream data, wherein the first group of stream data comprises stream data sent to a first service end by a binary group corresponding to a first IP address, the first characteristic information comprises the number of the stream data and the sending time of each piece of stream data, and the first service end is one of the M service ends and corresponds to a second IP address;

acquiring second characteristic information of a second group of stream data according to the K pieces of stream data, wherein the second group of stream data comprises stream data sent to a second server by a binary group corresponding to a first IP address and/or a binary group corresponding to a second IP address, the second characteristic information comprises the number of the stream data and the sending time of each piece of stream data, and the second server is one of the M servers;

and determining a second call chain according to the first IP address, the first service end and the second service end based on the incidence relation between the first group of stream data and the second group of stream data meeting a preset condition, wherein the second call chain is used for indicating that the first service end calls the second service end.

8. The method according to any one of claims 5 to 7, wherein the dividing the M servers into X server sets according to the Q first call chains comprises:

determining a third server as a core server based on the fact that the frequency of the third server appearing in the Q first call chains is greater than a third threshold, wherein the third server is one of the M servers;

9. The method of claim 8, further comprising:

and determining the fourth server as a core server based on the third number being greater than a fourth threshold.

10. The method according to any one of claims 5 to 9, wherein the dividing the M servers into N sets of servers according to the streaming data comprises:

and on the basis that the similarity between the fifth server and the sixth server is greater than a preset similarity, dividing the fifth server and the sixth server into a similar server set.

11. The method according to claim 10, wherein the obtaining the similarity between the fifth server and the sixth server according to the K pieces of flow data comprises:

acquiring feature data of the communication session between the fifth server and the H binary groups according to the K pieces of stream data, wherein the feature data is used for representing the characteristics of the communication session;

acquiring G binary groups communicated with a sixth server according to the K pieces of stream data, wherein G is a positive integer;

acquiring feature data of a communication session between the sixth server and the G binary groups according to the K pieces of stream data;

12. The method according to claim 10 or 11, wherein the fifth server belongs to a first server set and belongs to a similar server set, and the first server set is one of the X server sets;

after the dividing the fifth server and the sixth server into similar server sets, the method further includes:

13. An apparatus for analyzing streaming data, comprising:

the identification unit is used for identifying M service-side duplets from the duplets of the streaming data according to the streaming data, wherein K is a positive integer, and M is a positive integer;

and the set dividing unit is used for dividing the M service ends into N service end sets according to the streaming data, wherein the service end sets comprise one or more service ends, N is a positive integer, and if the service end sets comprise a plurality of service ends, the service ends have an association relationship.

14. An apparatus for analyzing streaming data, comprising: at least one processor and a memory, the memory storing computer-executable instructions executable on the processor, the terminal device performing the method of any one of claims 1-12 when the computer-executable instructions are executed by the processor.

15. A computer-readable storage medium storing one or more computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, the processor performs the method of any one of claims 1-12.