CN115658220A - Data processing method, equipment and computer readable storage medium - Google Patents
Data processing method, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115658220A CN115658220A CN202211255497.2A CN202211255497A CN115658220A CN 115658220 A CN115658220 A CN 115658220A CN 202211255497 A CN202211255497 A CN 202211255497A CN 115658220 A CN115658220 A CN 115658220A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- address
- processed
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the application discloses a data processing method, which comprises the following steps: acquiring a target isolation strategy for a target object; the target isolation strategy is used for controlling data transmission of the target object; determining, by a first application, a first operation to be performed on first to-be-processed data of a target object based on a target isolation policy; the first application is arranged on the control surface of the data processing equipment; executing a first operation on the first data to be processed through a second application; wherein the second application is arranged on a data plane of the data processing device. The embodiment of the application also discloses data processing equipment and a computer readable storage medium.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, device, and computer-readable storage medium.
Background
Kubernets, also called k8s, is an open source container cluster management system, which is used to implement automated container deployment, elastic container shrinkage, load balancing among containers, and the like. When a tenant deploys services in k8s, one or more pods are created, and in order to ensure the security of tenant data, the pods of different tenants are not intercommunicated. Generally, data output by a Container Network Interface (CNI) passes through a data filtering system (Internet Protocol tables), and an isolation policy for limiting Network data transmission between different Pod is converted into an iptables rule, and transmission of data output by the CNI is limited by a filter table on an input (input) chain and an output (forward) chain of the iptables. However, performance consumption of filtering all data through iptables is large, and iptables cannot be decided in an application scenario based on an extended Berkeley Packet Filter (eBPF) data plane and an OVS (OpenvSwitch) data plane, so that network isolation cannot be achieved, and an application range is narrow.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application are expected to provide a data processing method, a device, and a computer-readable storage medium, which solve the problems in the related art that performance consumption is large and an application range is narrow when filtering all data through iptables.
The technical scheme of the application is realized as follows:
a method of data processing, the method comprising:
acquiring a target isolation strategy for a target object; wherein the target quarantine policy is to control data transfer of the target object;
determining, by a first application, a first operation to be performed on first to-be-processed data of the target object based on the target isolation policy; the first application is arranged on the control surface of the data processing equipment;
executing the first operation on the first data to be processed through a second application; wherein the second application is provided on a data plane of the data processing device.
In the foregoing solution, the obtaining a target isolation policy for a target object includes:
acquiring the target isolation strategies aiming at the target objects of different clusters and generated on a management surface of the data processing equipment based on the priority of the isolation strategies; wherein each of the clusters comprises a plurality of objects.
In the foregoing solution, before determining, by the first application and based on the target isolation policy, a first operation to be performed on the first to-be-processed data of the target object, the method further includes:
and acquiring first data to be processed from the data plane through the first application.
In the above solution, the determining, based on the target isolation policy, a first operation to be performed on first to-be-processed data of the target object includes:
analyzing the first data to be processed to obtain a sending address and a destination address;
processing the sending address and the destination address based on the target quarantine policy to determine the first operation.
In the above solution, the processing the sending address and the destination address based on the target quarantine policy to determine the first operation includes:
determining a release address and an interception address corresponding to the target object from the target isolation strategy based on the sending address;
determining that the first operation is a clear operation if the destination address matches the clear address; wherein the releasing operation is used for sending the first data to be processed to the destination address;
determining that the first operation is an intercept operation if the destination address matches the intercept address.
In the above scheme, the method further comprises:
acquiring target data with the same sending address and/or the same isolation strategy from the first data to be processed, and storing the target data;
and counting the operation corresponding to the target data to obtain an operation set, and displaying the operation set through a management surface.
In the foregoing solution, the method further includes:
under the condition that the target object is determined to be changed by the first application, updating the target isolation strategy based on the changed target object;
determining a second operation of second data to be processed aiming at the changed target object based on the modified target isolation strategy;
and executing the second operation on the second data to be processed.
In the above scheme, the method further comprises:
determining address data corresponding to the changed target object;
updating the priority of the modified target quarantine policy based on the target address.
A data processing apparatus, the apparatus comprising: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is used for executing the data processing program in the memory so as to realize the steps of the data processing method.
A computer readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps of the data processing method described above.
The data processing method, the data processing device and the computer-readable storage medium provided by the embodiments of the application can acquire a target isolation policy for a target object, determine a first operation to be executed on first to-be-processed data of the target object based on the target isolation policy through a first application, and execute the first operation on the first to-be-processed data through a second application, so that the first operation to be executed on the first to-be-processed data is determined through the first application, and the second application executes the first operation, so that the transmission data can be decided without depending on iptables, the application range is wide, the performance consumption is low, and the problems that the performance consumption is high and the application range is narrow when all data are filtered through iptables in the related art are solved.
Drawings
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data processing system according to an embodiment of the present application;
fig. 3 is a schematic flowchart of another data processing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart illustrating issuing of an isolation policy in a data processing method according to an embodiment of the present application;
fig. 5 is a schematic view illustrating a flow of asset delivery in a data processing method according to an embodiment of the present application;
fig. 6 is a schematic flowchart illustrating issuing of an isolation policy in another data processing method according to an embodiment of the present application;
fig. 7 is a schematic flowchart of another data processing method according to an embodiment of the present application;
fig. 8 is a schematic flowchart illustrating intercepting data in a data processing method according to an embodiment of the present application;
fig. 9 is a schematic flow chart illustrating reporting of a network isolation condition in a data processing method according to an embodiment of the present application;
FIG. 10 is a schematic flow chart illustrating the processing of changed assets in a data processing method according to an embodiment of the present application;
fig. 11 is a schematic diagram illustrating that a data processing method provided in the embodiment of the present application is applied to multiple clusters;
fig. 12 is a schematic flowchart illustrating a data processing method applied to a generic container according to an embodiment of the present application;
fig. 13 is a schematic flowchart of network isolation performed on Pod in a data processing method according to an embodiment of the present application;
fig. 14 is a schematic flowchart illustrating a process of performing network isolation on a Node in a data processing method according to an embodiment of the present application;
fig. 15 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
It should be appreciated that reference throughout this specification to "an embodiment of the present application" or "an embodiment of the foregoing" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrase "in an embodiment of the present application" or "in the foregoing embodiment" appearing in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In a case where no specific description is given, the electronic device may execute any step in the embodiments of the present application, and the processor of the electronic device may execute the step. It should also be noted that the embodiment of the present application does not limit the sequence of the steps executed by the electronic device. In addition, the data may be processed in the same way or in different ways in different embodiments. It should be further noted that any step in the embodiments of the present application may be executed by the electronic device independently, that is, when the electronic device executes any step in the following embodiments, the electronic device may not depend on the execution of other steps.
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
An embodiment of the present application provides a data processing method, which may be applied to a data processing device, and as shown in fig. 1, the method includes the following steps:
Wherein the target isolation policy is used to control data transfer of the target object.
In this embodiment of the present application, a target object is an object that needs to be network isolated currently, the target object may be one or more objects, and the target object may be an entity of different levels included in one cluster or an entity of different levels included in different clusters; in one implementation, the target object may be a Node (Node), or the target object may be a sub-object in the Node, which may be a Pod; the target isolation policy is an isolation policy generated for the target object.
In one possible approach, a k8s cluster may be deployed on a data processing device, the k8s cluster including a plurality of nodes, each node including a plurality of pods, each Pod including a plurality of containers; as shown in fig. 2, k8s includes a management plane, a control plane, and a data plane; a User Interface (UI) is arranged on the management surface, so that a User can select a target object to be subjected to network isolation through the UI, or can add an isolation strategy, delete the isolation strategy, adjust the isolation strategy and the like through the UI; the management plane may deploy a management application to generate a target isolation policy based on a target object selected by a user, such as a management container based on a micro-isolation policy engine in the management end background shown in fig. 2, where the target isolation policy is an Access Control List (ACL); after the management plane of k8s generates the target isolation policy, the target isolation policy may be issued to the control plane through a Remote Procedure Call Protocol (RPC). In addition, after the management application generates the target isolation policy, the target isolation policy may be stored in a target form in the database, and in a feasible manner, the target isolation policy may be directly stored in a form of a character string in a relational database management system (MySQL).
The first application is arranged on the control plane of the data processing equipment.
In this embodiment of the present application, the first application may be configured to determine, based on the target isolation policy, an operation to be performed on first to-be-processed data of the target object, that is, to determine how to process the to-be-processed data of the object corresponding to the isolation policy based on the isolation policy, where the first application may be a micro-isolation Agent (Agent) or a firewall (Defender), and of course, the first application may also be another application, which is not limited in this embodiment of the present application. The first data to be processed is data which is currently sent to other addresses by the target object; the first operation is an operation performed on the first to-be-processed data, and the first operation may be release or interception, and the first operation may also be interception and reporting, or temporary non-processing, and the like.
In a possible manner, as shown in fig. 2, an Agent may be deployed in the control plane, so that a first operation (release or interception) performed on the first to-be-processed data of the target object may be determined by the Agent based on the target isolation policy. In addition, after the first operation is determined, the first operation may be issued to the data plane, and the data plane processes the first data to be processed.
And 103, executing a first operation on the first to-be-processed data through a second application.
Wherein the second application is arranged on the data side of the data processing device.
In this embodiment of the present application, a second application performs a corresponding operation on first data to be processed, where the second application may be an iptables; if the first operation is a release operation, the release operation is carried out on the first data to be processed, namely the transmission of the first data to be processed is allowed; if the first operation is interception, performing interception operation on the first data to be processed, namely not allowing the first data to be processed to be transmitted; if the first operation is interception and reporting, it is indicated that the first to-be-processed data is possibly dangerous data, and at this time, the first to-be-processed data is not only intercepted, but also needs to be reported to the management plane; if the first operation is a temporary non-processing operation, which indicates that it is not time to process the first to-be-processed data at this time, the release or interception may be performed on the first to-be-processed data after the target time period. In one possible approach, iptabiles on the data plane performs corresponding operations on the first to-be-processed traffic.
In the embodiment of the application, the first operation executed on the first to-be-processed data of the target object is determined through the first application of the control plane, the first operation is executed on the first to-be-processed data through the second application of the data plane, the first operation is determined through the control plane, the first operation is executed through the data plane, the data is decided without depending on iptables, the application range is wider, and the method and the device can be applied to application scenes based on an eBPF data plane and an OVS data plane.
It should be noted that the data processing method provided in the embodiment of the present application may be applied to a k8s system built on a physical machine, and may also be applied to a k8s system built on a virtual machine.
The data processing method provided by the embodiment of the application obtains the target isolation policy for the target object, determines the first operation executed on the first to-be-processed data of the target object based on the target isolation policy through the first application, and executes the first operation on the first to-be-processed data through the second application, so that the first operation executed on the first to-be-processed data is determined through the first application, and the second application executes the first operation, so that the transmitted data can be arbitrated without relying on iptables, the application range is wide, the performance consumption is low, and the problems that the performance consumption for filtering all data through iptables in the related art is large and the application range is narrow are solved.
Based on the foregoing embodiments, an embodiment of the present application provides a data processing method, which is shown in fig. 3 and includes the following steps:
Wherein each cluster comprises a plurality of objects, and the target isolation policy is used for controlling data transmission of the target objects.
In the embodiment of the present application, the objects in the cluster may be referred to as assets, and the assets may be nodes, devices, containers, pod, and the like. In a feasible manner, after the target isolation policy is generated, the priority of the target isolation policy may be set, so that the control plane may process the to-be-processed data of the corresponding target object in time based on the priority of the target isolation policy, thereby ensuring the security of the target object and the data transmission. The management plane can simultaneously generate the isolation strategies and the priority levels of the isolation strategies based on the objects selected by the multiple users, all the generated isolation strategies are issued to the control plane, and the control plane can process the transmitted data based on the priority levels of the isolation strategies. The management plane may be deployed on the current device, or may be deployed on other devices, and when deployed on other devices, the other devices need to be able to communicate with the current device, so that the target isolation policy generated by the management plane may be issued to each node in the cluster.
The method for issuing the target isolation strategy can be as follows: as shown in fig. 4, a user may add \ adjust \ change an isolation policy through a console of a management plane, store the isolation policy in a database in a target form through a Python module, and then issue the isolation policy to each cluster through a micro-isolation RPC server; maintaining all isolation policies and databases within the cluster through a policy module of the cluster; issuing an isolation strategy to each node in the cluster through node flow distribution of a management surface, wherein the issuing of the isolation strategy needs to pass through a micro-isolation RPC server; the micro-isolation policy module of each node control surface receives and processes the isolation policy, the policy management module of each node is used for managing and updating the local isolation policy, and the decision module is used for determining the operation executed on the data to be processed of the object corresponding to the isolation policy based on the isolation policy.
In other embodiments of the present application, assets in the clusters can be adjusted through the management plane, as shown in fig. 5, a user can add \ adjust \ change assets through a console of the management plane, and store the changed assets to an asset module, the asset module is used for maintaining all assets, and then sends the changed assets to each cluster through a micro-isolation RPC server; the strategy object module in the cluster is used for maintaining a strategy object (namely a specific isolation strategy) in the cluster, and issuing the isolation strategy to each node in the cluster through the node flow distribution of the management surface, wherein the issuing of the isolation strategy needs to pass through a micro-isolation RPC server; the micro-isolation policy object module of each node control surface manages and updates the local policy object, the policy management module of each node is used for managing and updating the local isolation policy, and the decision module is used for determining the operation executed on the to-be-processed data of the object corresponding to the isolation policy based on the isolation policy.
As shown in fig. 6, when the micro-isolation service is started, updating of all assets in the current device is triggered to obtain current latest asset information, where the asset information may include an asset identifier, an asset name, an asset state, and the like; after the latest asset information is obtained, issuing the full amount of the isolation strategy to all assets, and judging by an Agent of a control surface to which the assets belong based on the isolation strategy; or after the latest asset information is obtained, a network isolation object can be determined from the latest asset information in advance, and a target isolation strategy is issued to the network isolation object, so that the processing rate is increased, and the resource waste is reduced; if the network isolation object is a node, the data transmission in the node is controlled, so that the security of the data transmission in the node is guaranteed. Similarly, after the strategy is changed, the strategy also needs to be sent to the control surface, and the Agent of the control surface decides based on the isolation strategy; similarly, the network object or asset change needs to be sent to the control plane, and the Agent of the control plane arbitrates based on the isolation policy. In addition, if the assets are changed, the target isolation strategy is correspondingly changed, and the network isolation objects are also changed; but changes in the target isolation policy do not necessarily result in changes to the assets.
Wherein the first application is arranged at a control plane of the data processing device.
In this embodiment of the present application, the manner of acquiring the first to-be-processed data from the data plane through the first application may be: a first application sends a data acquisition request to a data plane, wherein the data acquisition request carries an iptables rule; after receiving the data acquisition request, the data plane acquires first data to be processed based on an iptables rule and sends the first data to be processed to a first application of the control plane; the iptables rule may be used to obtain first packet connection information of a Transmission Control Protocol (TCP) stream, or may be used to obtain information such as a network quintuple, and the iptables rule may be defined according to a service requirement.
In a feasible manner, as shown in fig. 2, the iptables rule is used to obtain a synchronization (syn) data packet, an Agent (i.e., a first application) of the control plane issues the iptables rule to the data plane, an nfqueue of the data plane obtains a syn data packet, and sends the obtained syn data packet to an Agent of the control plane; where nfqueue is used to delegate arbitration of data packets to user-state software.
In the embodiment of the application, the sending address is an issuing address of the first to-be-processed data, namely a source address; the destination address is the address of the destination to which the first data to be processed needs to be transferred; the first to-be-processed data may include address data such as Internet Protocol (IP) port information, a network five-tuple, and the like, so that after the first application of the control plane acquires the first to-be-processed data, the first to-be-processed data may be parsed to obtain a sending address and a destination address of the data.
In this embodiment, the target isolation policy may be for a plurality of objects, and the target isolation policy includes an operation in which a sending address and a destination address of each object correspond to each other. In a feasible manner, if the target isolation policy represents that all data sent by the address a can be transmitted to the address B, and all data sent by the address a is prohibited from being transmitted to the address C, that is, the first operation corresponding to the data sent by the address a to the address B is release; the first operation corresponding to the data sent by the address A to the address C is interception.
In a feasible manner, as shown in fig. 2, after the Agent of the control plane obtains the first to-be-processed data, the Agent parses the first to-be-processed data to obtain a sending address and a destination address corresponding to the first to-be-processed data, and then processes the sending address and the destination address according to the ACL rule, that is, arbitrates the first to-be-processed data to obtain a first operation of releasing or intercepting the first to-be-processed data.
Wherein the second application is arranged on the data side of the data processing device.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
According to the data processing method provided by the embodiment of the application, the first operation executed on the first to-be-processed data is determined through the first application, the second application executes the first operation, the transmission data can be judged without depending on the iptables, the application range is wide, the performance consumption is low, and the problems that the performance consumption is large and the application range is narrow when all data are filtered through the iptables in the related art are solved.
Based on the foregoing embodiments, an embodiment of the present application provides a data processing method, which is shown in fig. 7 and includes the following steps:
Wherein each cluster comprises a plurality of objects, and the target isolation policy is used for controlling data transmission of the target objects.
The data acquisition request is used for acquiring first to-be-processed data; the first application is arranged at a control plane of the data processing device.
And step 304, the data processing equipment determines a release address and an interception address corresponding to the target object from the target isolation strategy through the first application based on the sending address.
In the embodiment of the present application, the release address refers to an address to which data sent from a sending address can be transmitted, that is, the data sent to the release address can be released; the interception address is an address to which data sent by the sending address cannot be transmitted, that is, the data sent to the interception address is intercepted.
In the case where the destination address matches the release address, the data processing apparatus determines that the first operation is a release operation, step 305.
Wherein the clear operation is used to send the first data to be processed to the destination address.
In this embodiment, the destination address is matched with the release address, which indicates that the first to-be-processed data sent by the sending address can be transmitted to the destination address, and at this time, the first operation is a release operation, that is, the first to-be-processed data is sent to the destination address.
In step 306, in case the destination address matches the interception address, the data processing apparatus determines that the first operation is an interception operation.
In the embodiment of the present application, the destination address matches the interception address, which indicates that the first to-be-processed data sent by the sending address cannot be transmitted to the destination address, and at this time, the first operation is an interception operation, that is, the first to-be-processed data is not allowed to be transmitted to the destination address.
In a feasible manner, as shown in fig. 8, a syn packet may be obtained based on nfqueue in the k8s node, and then the content in the syn packet is parsed to obtain a sending address and a destination address, and then the sending address and the destination address are arbitrated by an isolation policy to determine to pass or intercept the transmitted data.
Wherein the second application is arranged on a data plane of the data processing device.
Based on the foregoing embodiment, in another embodiment of the present application, as shown in fig. 7, the data processing method may further include the following steps:
In the embodiment of the application, the target data may be data with the same sending address and the same isolation policy, may be data with the same sending address, and may also be data with the same isolation policy; the target data may be IP port information of a data packet; determining target data with the same sending address and/or the same isolation strategy in the first data to be processed, namely dividing the first data to be processed according to the sending address and/or the isolation strategy, wherein the obtained target data is the data of the sending address and/or the isolation strategy. In one possible approach, the packet IP port information of each data in the target data may be extracted and put into the cache.
In the embodiment of the application, the target data with the same sending address and/or the same isolation strategy is obtained, that is, the data with the same sending address and/or the same isolation strategy is summarized, so that the operation condition corresponding to the data sent by each sending address and the current port data are intercepted or released according to how many times the strategy is matched can be conveniently known.
In the embodiment of the application, the operation set is a set of corresponding operations corresponding to the target data; corresponding operations corresponding to the target data can be counted to obtain an operation set, namely, the first operation determined based on the target isolation strategy is counted, namely, the matching condition of the target isolation strategy is counted. The operation set is displayed through the management surface, so that a user can conveniently check the matching condition of the target isolation strategy and the data transmission condition of each target object. In addition, the cache data can be transmitted to the message application under the condition that the cache data is larger than the configuration value, and then the gathered data is reported to the management surface by monitoring the message application, so that a user can conveniently check the strategy matching condition and the data transmission condition.
As shown in fig. 9, an Agent obtains IP port information of a data packet of target data and puts the IP port information into a cache, determines to send the target data with the same address and/or the same isolation policy and counts corresponding operations corresponding to the target data to obtain an operation set, transmits the target data and the operation set to a message publishing/subscribing delivery platform (pulser) when the cache data is greater than a configuration threshold, and a management plane may monitor the pulser to write the target data and the operation set into a database (database, DB) and display the target data and the operation set through a UI interface, so that a user can know matching conditions of the isolation policy conveniently.
In step 310, in the case that the first application is adopted to determine that the target object is changed, the data processing device updates the target isolation policy based on the changed target object.
In the embodiment of the present application, the first application may also be used to detect whether the target object changes. Whether the target object is changed or not comprises the steps of adding a new sub-object in the target object, reducing the sub-object, changing the address of the sub-object, changing the position of the sub-object and the like. In one possible approach, the first application may monitor the target object through a monitoring mechanism; as shown in FIG. 10, agents can pass through; the list listening component (ListWacher) listens whether the target object has changed.
In this embodiment, the second data to be processed is data that needs to be transmitted by the changed target object. The second operation is an operation performed on second data to be processed; the second operation may be release or intercept. When the target object changes, the corresponding target isolation strategy needs to be updated, the corresponding target isolation strategy is updated at any time based on the changed target object, and the data to be processed of the changed target asset is processed based on the updated target isolation strategy, so that the safety of the target object is protected in real time, and the safety of data transmission is improved.
In the embodiment of the present application, the second operation is an operation performed on the second to-be-processed traffic data; the second operation may be release or intercept. As shown in fig. 10, when the target asset occurs and the target isolation policy is modified based on the changed target asset, the modified target isolation policy needs to be issued to iptables of the data plane at this time, then the current data to be processed is obtained, and the current data to be processed is arbitrated based on the modified target isolation policy, so as to obtain the second operation.
In the embodiment of the present application, a second operation may be performed on second to-be-processed data by a second application; if the second operation is releasing, the releasing operation is executed on the second data to be processed, namely the transmission of the second data to be processed is allowed; and if the second operation is interception, performing interception operation on the second data to be processed, namely not allowing the second data to be processed to be transmitted. In one possible embodiment, a second application of the data plane performs a corresponding operation on the second pending traffic.
In other embodiments of the present application, if the target object is changed, steps 313 to 314 may be further executed to update the priority of the modified target quarantine policy.
In the embodiment of the application, the address data corresponding to the changed target object can be determined by the first application; in a feasible manner, the first application may obtain address data corresponding to the changed target object from the micro-isolation policy object module.
In the embodiment of the application, the priority of the target isolation policy is updated based on the address data of the changed target object, and the data to be processed of the target object corresponding to the target isolation policy with higher priority is processed in time based on the priority of the target isolation policy, so that the security of data transmission is improved. In a feasible manner, if the IP address of the target object Pod1 is 10.0.0.2/24, and the IP of the Pod1 after the asset update is 10.0.0.10/24, at this time, the priority of the target quarantine policy corresponding to the Pod1 needs to be updated according to the IP address of the Pod1 after the change (i.e., 10.0.0.10/24), which is convenient for the first application to determine the second operation on the second to-be-processed data of the Pod1 based on the updated target quarantine policy, and to determine whether to release or intercept the second to-be-processed traffic based on the second operation.
As shown in fig. 10, when the Agent monitors that the asset in the target object changes through the listwatch, the Agent updates the target isolation policy based on the changed asset, issues an iptables rule to the updated target isolation policy through the Agent iptables module to obtain second data to be processed, determines a second operation to be performed on the second data to be processed based on the target isolation policy, and performs corresponding processing on the second data to be processed based on the second operation.
The data processing method provided in the embodiment of the present application may be applied to multiple systems or multiple clusters, as shown in fig. 11, a k8s cluster may be deployed on one virtual machine or physical machine, where the k8s cluster includes a control plane (Master) and two nodes, where the Master is responsible for management and control of the entire k8s cluster; each Node includes two pods and a secure container. A cluster of openshift or rancher, which includes two nodes, where each Node includes two pods and one secure container, may be deployed on another virtual or physical machine. It should be noted that, two clusters in fig. 11 are managed by one management container, the management container may be set in any one cluster, and fig. 11 is only illustrated by taking an example of deployment on a cluster formed by a cloud development platform (openshift) or a centralized identity authentication system (rancher). The management container in fig. 11 is a management plane, and is configured to obtain an isolation policy generated by a user for an object selected by two clusters in fig. 11, and issue the isolation policy to a security container in each cluster; the safety container is a control surface and is used for acquiring the TCP data packet by utilizing the iptables nfqueue, extracting the SYN data packet in the TCP data packet, matching the extracted data packet with an isolation strategy issued by the management container and determining whether to pass or intercept data. In addition, aiming at a User Datagram Protocol (UDP) data packet and an ICMP (Internet Control Message Protocol) data packet, after a User issues an isolation strategy in a management container, a safety container converts the data packet into an iptables rule and issues the iptables rule to a node, the CNI forwarded by a kernel is not relied on, the function of a k8s original component network proxy component (club-proxy) and the club-proxy used by the service (service) of a part of OVS data plane are not relied on; the kube-proxy is a core component of k8s, is deployed on each Node, and is used for realizing an important component of a communication and load balancing mechanism of the k8s Service. In addition, k8s has a set of k8s standard network policy (network policy), and the management plane can realize a network policy management and automation capability by referring to a super system tool (sysdig). The management container is responsible for being connected with a multi-heterogeneous and multi-cluster platform, network segment information distributed by a cluster is obtained and is issued to the defender, the defender issues iptables rules according to the network segment information provided by the management container, and the nfqueue extracts connected data packets and is matched with an isolation strategy issued by a user in the management container.
FIG. 12 is a flowchart of a method for network isolation of a generic container to Pod east and west, where a Master issues an isolation policy to a Defender via an Application Programming Interface-Server (API-Server); the Defender can acquire a syn data packet of Pod through the iptables nfqueue, as shown in fig. 13, match the acquired syn data packet based on an isolation strategy, and determine whether to release or intercept the data; wherein, the iptables rule can be: the Post flag in the Pod includes an output release address and an output intercept address, and both the release address and the intercept address may be a subnet (subnet section allocated by CNI) allocated to a container on the Node, a Node IP segment, a container (docker 0) segment, a local loop interface (lo) address, an IP segment configured by a management plane, and a group (Cluster) IP segment.
The data processing method provided by the embodiment of the application can also be used for network isolation of the privileged container to the host, as shown in fig. 14, the Defender can acquire a syn data packet of the Node through the iptables nfqueue, and matches the acquired syn data packet based on the isolation policy to determine whether to pass or intercept the data; wherein, the iptables rule can be: the Node internal input table comprises a release address and an interception address, wherein the release address and the interception address can be subnets distributed by a container on a Node, an IP section of the Node and a local lo address; the Node internal output table comprises a source IP address, a Node IP section and a local lo address, wherein the source IP address is a subnet distributed by a container on a Node.
The data processing method provided by the embodiment of the application can be applied to micro-isolation capability in a cloud native scene to prohibit access between Pod and Pod, access between Pod and Node, and access between Pod and service, support isolation of Pod of a cluster crossing Node, dock other platforms through a management container, isomerize various platforms, acquire network segments allocated by each platform, issue the network segments allocated by the platforms to a safety container, achieve isomerous multiple platforms, support TCP/UDP/ICMP (transmission control protocol)/IP (Internet control protocol)/Internet protocol, and multiplex k8s network policy capability for a user to select; the PRE-match table in the Pod includes an input source address (i.e. a sending address), where the source address may be a subnet allocated to a container on the Node (a subnet allocated to a CNI), a Node IP segment, a docker0 network segment, a local lo address, an IP segment configured for a management plane, a cluster IP segment, and the like.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
According to the data processing method provided by the embodiment of the application, the first operation executed on the first to-be-processed data is determined through the first application, the second application executes the first operation, the transmission data can be judged without depending on the iptables, the application range is wide, the performance consumption is low, and the problems that the performance consumption is large and the application range is narrow when all data are filtered through the iptables in the related art are solved.
Based on the foregoing embodiments, an embodiment of the present application provides a data processing apparatus, which may be applied to the data processing method provided in the embodiments corresponding to fig. 1, 3 and 7, and as shown in fig. 15, the data processing apparatus 4 may include: a processor 41, a memory 42, and a communication bus 43, wherein:
the communication bus 43 is used for realizing communication connection between the processor 41 and the memory 42;
the processor 41 is configured to execute a data processing program in the memory 42 to implement the following steps:
acquiring a target isolation strategy for a target object; the target isolation strategy is used for controlling data transmission of the target object;
determining, by a first application, a first operation to be performed on first to-be-processed data of a target object based on a target isolation policy; the first application is arranged on the control surface of the data processing equipment;
executing a first operation on the first data to be processed through a second application; wherein the second application is arranged on the data side of the data processing device.
In other embodiments of the present application, the processor 41 is configured to execute the data processing program in the memory 42 to obtain the target isolation policy for the target object, so as to implement the following steps:
acquiring target isolation strategies aiming at target objects of different clusters and generated on a management surface of data processing equipment based on the priority of the isolation strategies; wherein each cluster comprises a plurality of objects.
In other embodiments of the present application, the processor 41 is configured to execute the first operation of the data processing program in the memory 42, and determine the first operation to be executed on the first to-be-processed data of the target object based on the target isolation policy, and further implement the following steps:
and acquiring first data to be processed from the data plane through the first application.
In other embodiments of the present application, the processor 41 is configured to execute a target isolation policy based on the data processing program in the memory 42 to determine a first operation to be performed on first to-be-processed data of a target object, so as to implement the following steps:
analyzing the first data to be processed to obtain a sending address and a destination address;
the sending address and the destination address are processed based on the target quarantine policy, and a first operation is determined.
In other embodiments of the present application, processor 41 is configured to execute a target quarantine policy of a data processing program in memory 42 to process the send address and the destination address to determine a first operation to implement the steps of:
determining a release address and an interception address corresponding to the target object from the target isolation strategy based on the sending address;
determining that the first operation is a release operation in a case where the destination address matches the release address; the releasing operation is used for sending the first data to be processed to the destination address;
in the event that the destination address matches the intercept address, determining that the first operation is an intercept operation.
In other embodiments of the present application, the processor 41 is configured to execute the data processing program in the memory 42 to further implement the following steps:
acquiring target data with the same sending address and/or the same isolation strategy from the first data to be processed, and storing the target data;
and counting the operation corresponding to the target data to obtain an operation set, and displaying the operation set through a management surface.
In other embodiments of the present application, the processor 41 is configured to execute the data processing program in the memory 42 to further implement the following steps:
under the condition that the target object is determined to be changed by adopting the first application, updating a target isolation strategy based on the changed target object;
determining a second operation aiming at second data to be processed of the changed target object based on the modified target isolation strategy;
and executing a second operation on the second data to be processed.
In other embodiments of the present application, the processor 41 is configured to execute the data processing program in the memory 42 to further implement the following steps:
determining address data corresponding to the changed target object;
updating the priority of the modified target quarantine policy based on the target address.
It should be noted that, for specific descriptions of steps executed by the processor, reference may be made to the data processing method provided in the embodiments corresponding to fig. 1, 3, and 7, and details are not described here again.
According to the data processing device provided by the embodiment of the application, the first operation executed on the first data to be processed is determined through the first application, the first operation is executed through the second application, the transmission data can be judged without depending on the iptables, the application range is wide, the performance consumption is low, and the problems that the performance consumption is large and the application range is narrow when all data are filtered through the iptables in the related art are solved.
Based on the foregoing embodiments, embodiments of the present application provide a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement the steps of the data processing method provided by the embodiments corresponding to fig. 1, 3 and 7.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.
Claims (10)
1. A method of data processing, the method comprising:
acquiring a target isolation strategy for a target object; wherein the target quarantine policy is to control data transfer of the target object;
determining, by a first application, a first operation to be performed on first to-be-processed data of the target object based on the target isolation policy; the first application is arranged on the control surface of the data processing equipment;
executing the first operation on the first data to be processed through a second application; wherein the second application is provided on a data plane of the data processing device.
2. The method of claim 1, wherein obtaining the target isolation policy for the target object comprises:
acquiring the target isolation strategies aiming at the target objects of different clusters and generated on a management surface of the data processing equipment based on the priority of the isolation strategies; wherein each of the clusters comprises a plurality of objects.
3. The method of claim 1, wherein prior to determining, by the first application, the first operation to perform on the first to-be-processed data of the target object based on the target isolation policy, further comprising:
and acquiring first data to be processed from the data plane through the first application.
4. The method of claim 1, wherein the determining a first operation to perform on the first to-be-processed data of the target object based on the target isolation policy comprises:
analyzing the first data to be processed to obtain a sending address and a destination address;
processing the sending address and the destination address based on the target quarantine policy to determine the first operation.
5. The method of claim 4, wherein the processing the sending address and the destination address based on the target quarantine policy to determine the first operation comprises:
determining a release address and an interception address corresponding to the target object from the target isolation strategy based on the sending address;
determining that the first operation is a clear operation if the destination address matches the clear address; wherein the releasing operation is used for sending the first data to be processed to the destination address;
determining that the first operation is an intercept operation if the destination address matches the intercept address.
6. The method of claim 5, further comprising:
acquiring target data with the same sending address and/or the same isolation strategy from the first data to be processed, and storing the target data;
and counting the operation corresponding to the target data to obtain an operation set, and displaying the operation set through a management surface.
7. The method of claim 1, further comprising:
under the condition that the target object is determined to be changed by the first application, updating the target isolation strategy based on the changed target object;
determining a second operation of second data to be processed aiming at the changed target object based on the modified target isolation strategy;
and executing the second operation on the second data to be processed.
8. The method of claim 7, further comprising:
determining address data corresponding to the changed target object;
updating the priority of the modified target quarantine policy based on the target address.
9. A data processing apparatus, characterized in that the apparatus comprises: a processor, a memory, and a communication bus;
the communication bus is used for realizing communication connection between the processor and the memory;
the processor is configured to execute the data processing program in the memory to implement the steps of the data processing method according to any one of claims 1 to 8.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores one or more programs which are executable by one or more processors to implement the steps of the data processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211255497.2A CN115658220A (en) | 2022-10-13 | 2022-10-13 | Data processing method, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211255497.2A CN115658220A (en) | 2022-10-13 | 2022-10-13 | Data processing method, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115658220A true CN115658220A (en) | 2023-01-31 |
Family
ID=84987561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211255497.2A Pending CN115658220A (en) | 2022-10-13 | 2022-10-13 | Data processing method, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115658220A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116488945A (en) * | 2023-06-20 | 2023-07-25 | 杭州默安科技有限公司 | Container network isolation method and system |
-
2022
- 2022-10-13 CN CN202211255497.2A patent/CN115658220A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116488945A (en) * | 2023-06-20 | 2023-07-25 | 杭州默安科技有限公司 | Container network isolation method and system |
| CN116488945B (en) * | 2023-06-20 | 2023-09-15 | 杭州默安科技有限公司 | Container network isolation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11539665B2 (en) | Rule swapping in a packet network | |
| US20230421590A1 (en) | Rule-Based Network-Threat Detection | |
| US10411951B2 (en) | Network policy conflict detection and resolution | |
| US8234361B2 (en) | Computerized system and method for handling network traffic | |
| CN113014571B (en) | Method, device and storage medium for processing access request | |
| US10567441B2 (en) | Distributed security system | |
| CN114041276A (en) | Security policy enforcement and visibility for network architectures that mask external source addresses | |
| US20200204520A1 (en) | Virtual routing and forwarding (vrf)-aware socket | |
| EP2780815A2 (en) | Selective ip address allocation for probes that do not have assigned ip addresses | |
| US8713306B1 (en) | Network decoys | |
| CN115658220A (en) | Data processing method, equipment and computer readable storage medium | |
| CN106411852B (en) | Distributed terminal access control method and device | |
| US20190005100A1 (en) | Centralized state database storing state information | |
| CN110505243A (en) | The processing method and processing device of network attack, storage medium, electronic device | |
| CN110868392A (en) | Block chain safety control method and device based on SDN and block chain network | |
| WO2022241939A1 (en) | Network security management method and computer device | |
| CN113992412B (en) | Implementation method of cloud native firewall and related equipment | |
| CN116781301A (en) | Cross-namespace container security protection method, device, equipment and medium | |
| CN115834190A (en) | Host management and control method, device, equipment and storage medium | |
| CN114710491A (en) | Protection method of database cluster, database firewall and medium | |
| CN118713921A (en) | Network security management method, device, equipment and storage medium | |
| CN118400161A (en) | IP mask filtering method and device for data message and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |