WO2021034441A1 - Détection d'intrus pour un réseau - Google Patents

Détection d'intrus pour un réseau Download PDF

Info

Publication number
WO2021034441A1
WO2021034441A1 PCT/US2020/043080 US2020043080W WO2021034441A1 WO 2021034441 A1 WO2021034441 A1 WO 2021034441A1 US 2020043080 W US2020043080 W US 2020043080W WO 2021034441 A1 WO2021034441 A1 WO 2021034441A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
communications
client
data source
service
Prior art date
Application number
PCT/US2020/043080
Other languages
English (en)
Inventor
Manav Ratan Mital
Srinivas Nageswarrao Vadlamani
Pramod Chandraiah
Pedro Henrique Bragioni Las-Casas
Kaizen Navid Towfiq
Timothy Do Nguyen
Original Assignee
Cyral Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/548,732 external-priority patent/US11477196B2/en
Priority claimed from US16/900,619 external-priority patent/US11477217B2/en
Application filed by Cyral Inc. filed Critical Cyral Inc.
Publication of WO2021034441A1 publication Critical patent/WO2021034441A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action

Definitions

  • FIG. 1 is an exemplary embodiment of a system including a protective layer at the data source.
  • FIG. 2 is another exemplary embodiment of a system including a protective layer at the data source.
  • FIG. 3 is another exemplary embodiment of a system including a protective layer at the data source.
  • FIG. 4 is a flow chart depicting an exemplary embodiment of a method for authenticating a client for a data source.
  • FIG. 5 is a flow chart depicting an exemplary embodiment of a method for performing services for a client for a data source.
  • FIG. 6 is a flow chart depicting an exemplary embodiment of a method for performing multi-factor authentication for a client for a data source.
  • FIG. 7 is a flow chart depicting an exemplary embodiment of a method for performing federated identity management for a client for a data source.
  • FIG. 8 is a flow chart depicting another exemplary embodiment of a method for authenticating a client for a data source using federated identity management.
  • FIG. 9 is a flow chart depicting an exemplary embodiment of a method for analyzing and logging information related to queries of a data source.
  • FIG. 10 is a diagram depicting an exemplary embodiment of an abstract syntax tree.
  • FIGS. 11 A and 1 IB are flow charts depicting exemplary embodiments of methods for utilizing tokenization and/or encryption of sensitive data.
  • FIGS. 12A and 12B are flow charts depicting exemplary embodiments of methods for providing client information and for performing behavioral baselining for clients.
  • FIG. 13 is a flow chart depicting an embodiment of a method for detecting intruders to a network.
  • FIG. 14 is a flow chart depicting an embodiment of a method for detecting intruders to a network.
  • FIG. 15 is a flow chart depicting an embodiment of a method for detecting intruders to a network.
  • FIG. 16 is a diagram depicting an embodiment of a system for detecting intruders to a network.
  • FIG. 17 is a flow chart depicting an embodiment of a method for detecting intruders to a network.
  • FIG. 18 is a flow chart depicting an embodiment of a method for detecting intruders to a network.
  • the invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor.
  • these implementations, or any other form that the invention may take, may be referred to as techniques.
  • the order of the steps of disclosed processes may be altered within the scope of the invention.
  • a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task.
  • the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
  • the systems and methods described herein provide a protection layer, or sidecar, that resides at and functions as a secure perimeter around a data source.
  • Clients e.g. applications and/or end users
  • Compromised applications may be denied access to the data source.
  • This approach is in contrast to security measures that place a perimeter around the entire organization including the data source and applications.
  • the methods and systems may also track an application’s accesses of the data source to determine a context/behavioral baseline for the application. For example, the type of session, browser, APIs, IP address, query attributes etc. may be used to determine a behavioral baseline for accesses of the data source.
  • Subsequent accesses are compared to the baseline to determine whether a current query matches the baseline. If not, additional validation/defense mechanisms may be employed.
  • Application identity, or application fingerprinting may be incorporated into this behavioral baselining.
  • data masking may also be employed. In data masking, sensitive data are replaced by tokens or format preserving encryption having a similar form but without intrinsic meaning. For example, credit card or social security numbers may be replaced by a different number having the same number of digits. Tokens may also have an expiration time, requestor identification and other data used to track tokens’ use. Tokens are used unless/until the sensitive data are needed (e.g. credit card number needed to process a transaction). If the sensitive data are needed, tokens are presented to the data vault and validated to obtain the sensitive data.
  • a sidecar functions as a secure perimeter around a data source.
  • the sidecar includes a dispatcher and a number of services that communicate with the dispatcher.
  • the dispatcher may be a Layer 4 component that intercepts accesses of the data source. Both incoming accesses and outgoing data may be intercepted by the dispatcher.
  • the dispatcher is data agnostic so can be deployed across a variety of data sources.
  • the dispatcher may hold packets (step mode) or immediately forward packets to/from the data source (stream mode) but does not inspect the packets.
  • the dispatcher also provides packets to the services. The services perform deeper inspection of the packets intercepted by the dispatcher.
  • Services may provide one or more functions including but not limited to: authentication of the user/requester of the data source, query analysis, multifactor authentication, tokenization of data, rewriting, caching and behavioral baselining (using collectors), federated identity management and other functionality.
  • services may include Layer 7 (application layer) components.
  • Layer 7 application layer
  • the service may inspect the user’s credentials (e.g. user id and password and/or certificate) for that data source and only allow access to the data source if the user’s credentials are validated.
  • One or more of these services can be used together. This approach is in contrast to security measures that place a perimeter around the entire organization including the data source and applications or which attempt to manage the data source/organization using Layer 7 components.
  • collectors may be used at some/all of the clients using the data repository/data source. Collectors can be present for some or all clients having applications issuing queries to the data source. For example, collectors can be attached to applications for end users as well as in microservices utilized by end users.
  • a collector intercepts a communication (e.g. a query or method call) from the application and examines the state of the application. For example, the type of session, get/put/post/delete commands, APIs, IP address, query attributes, method calls, order of queries and/or other information may be detected by the collector. These represent the context of the query, or state of the application/client issuing the query. The collectors attach this context to the query.
  • microservices/multiple applications pass a query before the query is sent to a database.
  • one microservice may issue a query and pass the query to a second microservice.
  • the second microservice may add to the query or issue an additional query.
  • the collector for each microservice/application can obtain the context from that microservice/application.
  • the collector also attaches the context from that microservice/application to the query/queries.
  • multiple contexts may be obtained from multiple microservices/applications for a single access of a data source.
  • the query and context are passed to the sidecar when the data source is accessed. Over multiple accesses, the context can be used by the sidecar (e.g.
  • a service to determine the behavior (sequence of states/contexts) for each application’s accesses of the data source.
  • a model of the behavior e.g. a Hidden Markov Model
  • Subsequent accesses are compared to the behavioral baseline to determine whether a current query matches the behavioral baseline. If not, additional validation/defense mechanisms may be employed. For example, access to the data source can be refused and/or multifactor authentication invoked.
  • Collectors and/or behavioral baselining may be used in conjunction with other services.
  • Requests regarding sensitive data can be sent to a tokenization/encryption service by the dispatcher.
  • the service replaces sensitive data with tokens/format preserving encryption (FPE), or analogous data that is without intrinsic meaning.
  • FPE tokens/format preserving encryption
  • What is used is configurable on the attribute level. For example, for credit card numbers of a user, FPE can be used. For the social security number of the same user, a token can be used.
  • the request to store sensitive information can be provided to the service and the value encrypted before storage in the data store. A request to obtain the number can be provided to the service which can decrypt/detokenize if the requester is authorized.
  • the token/FPE can be provided in response to the request.
  • Tokens FPE can have an expiration time, requestor identification or other information. Tokens are used unless/until the sensitive data are needed (such as a credit card number needed to process a transaction). If the sensitive data are needed, tokens are presented to the data vault and validated to obtain the sensitive data. This validation may require additional, more trusted mechanisms for ensuring the security of the sensitive data. Tokenization can be used in conjunction with behavioral baseline/collectors, multifactor authentication and/or other services.
  • Multifactor authentication may also be performed using a service.
  • the dispatcher is placed in step mode, to hold packets instead of directly forwarding to a data source.
  • the dispatcher also provides packets to the service, which controls multi-factor authentication.
  • the service performs multifactor authentication (e.g. requiring a code to be sent to a separate device and input) or accesses a third party multifactor authentication service.
  • the dispatcher is placed in stream mode, allowing the user to communicate with the data source.
  • the data agnostic dispatcher can provide multifactor authentication across a number of different data sources, such as databases that do not traditionally support multifactor authentication.
  • the dispatcher can forward requests/queries to an analyzer service that performs query analysis.
  • the analyzer service examines/parses the queries and emits a structure that describes the query.
  • the structure is an abstract syntax tree.
  • the structure can be used to determine whether the query represents an attack, (e.g. to obtain unauthorized information) to determine whether the query will adversely affect performance.
  • queries which include wildcards or tautologies in users’ names can result in exfiltration of sensitive information. Queries that are missing a where clause can adversely affects database performance. Such queries may be identified. The packet including the query can then be stopped or the query can be canceled. Further, the structure emitted by the query analyzer may be used to log or otherwise store information about the queries.
  • the structure may be used to identify and log features of the query that may be of interest. Such information may be used in conjunction with analytics to examine use of the data source.
  • the information regarding queries that is logged can be indexed and may include only information previously identified as sensitive. For example, the tables, columns in tables, and data entries corresponding to the identified sensitive information may be logged. Further, such information redacted so that certain information, such as names and/or social security numbers, are not included in the log.
  • Another service provided may include query rewriting, which may be used with query analysis. Queries that may adversely affect performance can be rewritten. For example, a limit may be added so that too many results are not returned, a tautology removed, and/or other portions of the query rewritten to improve performance.
  • a service can be used for federated identity management (for example via lightweight directory access protocol (LDAP)).
  • LDAP lightweight directory access protocol
  • An end user requests access to the data repository/database and provides the directory access credentials via the sidecar.
  • the dispatcher provides the request to a service controlling federated identity management.
  • the sidecar/service determines whether the end user exists on a directory access server (e.g. an LDAP server), authenticates the end user’s credentials on the directory access server and determines the groups to which the end user belongs.
  • the service uses the group information to access the data repository as a proxy for the end user.
  • a method and system for performing at least one service include receiving a communication for a data source at a sidecar.
  • the sidecar includes a dispatcher and at least one service.
  • the dispatcher receives the communication and is data agnostic.
  • the method and system also include providing the communication from the dispatcher to the data source and to the at least one service.
  • the at least one service inspects the communication and may perform additional functions.
  • the dispatcher is an open systems interconnection (OSI) Layer 4 dispatcher and the service(s) include OSI Layer 7 service(s).
  • the method and system also include performing at least one function by the at least one service based on the communication.
  • the at least one function may include one or more of authentication services, multifactor authentication, tokenization of data, encryption of data, query analysis, behavioral baselining, and federated identity management.
  • the communication may be generated at a client and service(s) may include an authentication service.
  • the method and system further include determining whether the client is authorized to access the data source using the authentication service and preventing access to the data source if the client is not authorized to access the data source.
  • the communication may be provided to the data source only if the authentication service determines the client is authorized to access the data source.
  • the communication to the data source from the dispatcher and recalled before processing by the data source if the authentication service determines the client is not authorized to access the data source.
  • the communication is generated at a client and includes a first communication.
  • the service(s) include an authentication service the dispatcher is in a step mode for the first communication from the client.
  • the first communication may be provided to the authentication service without forwarding the first communication to the data source (step mode).
  • the authentication service may use the first communication to determine whether the client is authorized to access the data source. If it is determined that the client is not authorized to access the data source, access to the data source by the client may be prevented by terminating a connection to the client. If it is determined that the client is authorized, then the first communication is provided from the dispatcher to the data source.
  • the dispatcher is also placed in a stream mode. Upon receiving additional communication(s) from the client, the dispatcher, automatically forwards the at least one additional communication from the dispatcher to the data source.
  • a method and system that may utilize behavioral baselining include receiving, at the sidecar, a communication and a context associated with the communication from a client.
  • the communication is for a data source.
  • the sidecar includes the dispatcher and the service.
  • the method and system also include providing the context from the dispatcher to the service.
  • the method and system use the service to compare the context to a behavioral baseline for the client.
  • the behavioral baseline incorporates a plurality of contexts previously received from the client.
  • the method and system also permit the client access the data source only if the context is consistent with the behavioral baseline.
  • the behavioral baseline may also be updated based on the context.
  • the communication is intercepted at the client using a collector.
  • the collector determines the context of the client and attaches the context to the communication.
  • the method and system include receiving, at the dispatcher and from the client, additional communication(s) and additional context/s) associated with the additional communication(s).
  • the additional communication(s) are for the data source.
  • the additional context/s) are provided from the dispatcher to the service.
  • the service compares the context and additional context(s) to the behavioral baseline for the client.
  • the client is allowed to access the data source only if the context and the additional context/s) are consistent with the behavioral baseline.
  • the method and system also include receiving additional context/s) associated with the communication from additional client(s).
  • the additional context/s) are also provided from the dispatcher to the service. In such embodiments, the service compares the context and the additional context/s) to the behavioral baseline for the client.
  • a method and system for anonymizing data include receiving, at the sidecar, a request to store data in a data source.
  • the method and system also include providing the request from the dispatcher to the at least one service and anonymizing, at the service(s), the data to provide anonymized data.
  • the anonymized data may be provided to the data source.
  • the request includes the data and additional data.
  • the anonymizing further includes identifying the data to be anonymized.
  • the anonymizing may include tokenizing and/or encrypting the data.
  • the tokenizing and/or encrypting may be selected based upon a policy for the data.
  • the encryption uses format preserving encryption.
  • the method and system further include receiving, from a requester, an access request for the data.
  • the requester is authenticated. If the requester is authorized to receive the data, the method and system include de-anonymizing the anonymized data and providing de-anonymized data to the requester. If the requester is authorized to receive the anonymized data, the method and system include providing the anonymized data.
  • a method and system for performing authentication are described. The method and system include receiving, from a client, a communication for a data source at the sidecar. The communication is provided from the dispatcher to the service. The service determines whether the client is authorized to access the data source. In some embodiments, the service determines whether the client is authorized by calling a multi- factor authentication (MFA) utility.
  • MFA multi- factor authentication
  • the method and system also include receiving, from the MFA utility, a success indication.
  • the success indication indicates whether authentication by the MFA utility is successful.
  • the MFA utility is a third-party MFA utility.
  • the method and system may also include preventing access to the data source if the success indication indicates that the authentication is unsuccessful. Preventing access may include providing the communication to the data source from the dispatcher and recalling the communication before processing by the data source if the authentication indication indicates the authentication is unsuccessful.
  • the communication includes a first communication and the dispatcher is in a step mode for the first communication. In such embodiment, the preventing access may include providing the first communication to the service without forwarding the first communication to the data source.
  • Preventing access to the data source may include terminating a connection to the client if the success indication indicates that the authentication is unsuccessful.
  • the first communication may be forwarded from the dispatcher to the data source if the success indication indicates that the authentication is successful.
  • the dispatcher may be placed in a stream mode if the success indication indicates that the authentication is successful.
  • At least one additional communication from the client may be received at the dispatcher. Such additional communications are automatically forwarded from the dispatcher to the data source if the success indication indicates that the authentication is successful.
  • the dispatcher is an open systems interconnection (OSI) Layer 4 dispatcher and wherein the at least one service includes at least one OSI Layer 7 service.
  • OSI open systems interconnection
  • a method and system for performing query analysis include receiving a query for a data source at the sidecar.
  • the method and system also include providing the query from the dispatcher to the data source and to the service.
  • the query is analyzed using the service.
  • analyzing the query further includes parsing the query, providing a logical structure based on the query and analyzing the logical structure.
  • a log is provided based on the logical structure.
  • the log may also be analyzed.
  • the method and system may include receiving at least one of an identification of sensitive data and sensitive data policies.
  • providing the log further includes redacting a portion of the log based on the sensitive data policies and the identification of the sensitive data.
  • the method and system may prevent access to the data source if the analysis of the query indicates the query is an attack. In some embodiments, the method and system rewrite the query if the analysis of the query indicates the query adversely affects performance. In some embodiments, the analysis of the query indicates that data for a response to the query is in a cache. In such embodiments, the method and system may provide the data from the cache.
  • a method and system for performing federated identity management include receiving a communication for a data source at a sidecar.
  • the sidecar includes a dispatcher and a service.
  • the dispatcher receives the communication and is data agnostic.
  • the communication corresponds to end user credentials for an end user.
  • the method and system include providing the communication from the dispatcher to the data source and to the service.
  • the method and system also use the service to authenticate the end user based on the end user credentials and utilizing federated identity management.
  • the method and system also include accessing the data source by the sidecar as a proxy for the end user if the end user is authenticated using the federated identity management.
  • the end user activities for the data source may also be logged.
  • authenticating the end user includes providing the end user credentials to a federated identity management data source.
  • the federated identity management data source may be searched for group(s) to which the end user belongs.
  • a read only binding of the service to the federated identity management data source is performed.
  • the federated identity management data source may be searched for the end user using the read only binding. If the end user is authenticated, the method and system may include binding the service to the federated identity management data source as a proxy for the end user. Access to the data source may be prevented if the end user is not authenticated utilizing the federated identity management.
  • FIG. 1 is a diagram depicting an exemplary embodiment of a system 100 utilizing a protective layer between clients and data sources.
  • System 100 includes data sources 102 and 104, clients 106-1, 106-2 and 106-3 (collectively clients 106) and sidecar 110. Although two data sources 102 and 104, three clients 106 and one sidecar 110 are shown, in another embodiment, different numbers of data sources, clients, and/or sidecars may be used.
  • Data sources 102 and 104 may be databases, data stores, data vaults or other data repositories.
  • Clients 106 may be computer systems for end users and/or include applications which provide requests, or queries, to data sources 102 and 104.
  • Clients 106 may be part of the same organization as the data sources 102 and 104 or may be outside users of data sources 102 and 104.
  • clients 106 and data sources 102 and 104 may be part of the same business organization coupled by an internal network.
  • clients 106 may be outside users of data sources 102 and 104 connected to sidecar 110 and/or data sources 102 and/or 104 via the Internet or other external network.
  • some clients 106 may be external users of data sources 102 and 104 while other clients 106 are part of the same organization as data sources 102 and 104.
  • Sidecar 110 provides a protective layer between clients 106 and data sources
  • Sidecar 110 is configured such that its operation is data agnostic. Thus, sidecar 110 may be used with data sources 102 and 104 that have different platforms, are different databases, or are otherwise incompatible. Sidecar 110 is so termed because although depicted as residing between clients 106 and data sources 102 and 104, sidecar 110 may be viewed as enclosing, or forming a secure perimeter around data sources 102 and 104. Stated differently, clients 106 cannot bypass sidecar 110 in order to access data sources 102 and 104 in at least some embodiments. For example, a security group may be created for data sources 102 and 104. Dispatcher 112/sidecar 110 may be the only member of the security group.
  • clients 106 may access data sources 102 and 104 only through sidecar 110.
  • Clients 106 connecting to sidecar 110 may be internal or external to an organization. Therefore, sidecar 110 need not reside at the perimeter of an organization or network. Instead, sidecar 110 may reside at data sources 102 and 104. Stated differently, sidecar 110 may provide the final or only security for requests for data source 102 and 104 and need not provide security for other components of the organization. Thus, requests made by clients 106 may be passed directly from sidecar 110 to data sources 102 and 104 via a network.
  • Sidecar 110 provides security and other services for data sources 102 and 104 and clients 106.
  • sidecar 110 includes dispatcher 112 and services 114-1 and 114-2 (collectively services 114).
  • Dispatcher 112 is data agnostic and in some embodiments is a transport layer component (e.g. a component in Layer 4 of the Open Systems Interconnection (OSI) model). Dispatcher 112 thus performs limited functions and is not a Layer 7 (application layer) component.
  • dispatcher 112 receives incoming communications from clients 106.
  • a communication includes a request, a query such as a SQL query, or other transmission from clients 106 to access data source 102 or 104.
  • Dispatcher 112 also provides the requests to the appropriate data source(s) 102 and/or 104 and the appropriate service(s) 114-1 and/or 114-2. However, dispatcher 112 does not inspect incoming communications from clients 106 other than to identify the appropriate data source(s) 102 and/or 104 and corresponding service(s) 114 for the communication. Dispatcher 112 does not make decisions as to whether communications are forwarded to a data source or service. For example, a communication from a client 106 may include a header indicating the data source 102 desired to be accessed and a packet including a query.
  • dispatcher 112 may inspect the header to identify the data source 102 desired to be accessed and forwards the packet to the appropriate data source 102.
  • Dispatcher 112 also provides the packet to the appropriate service(s) 114.
  • dispatcher 112 does not perform deep inspection of the packet. Instead, the appropriate service(s) inspect the packet.
  • dispatcher 112 provides the communication to the appropriate service(s) 114 by storing the packet and providing to service(s) 114 a pointer to the storage location.
  • dispatcher 112 holds communications (e.g. packets) while service(s) 114 perform their functions. In other embodiments, dispatcher 112 directly forwards the communications to data source(s) 102 and/or 104 and services 114 separately perform their functions. In some embodiments, whether dispatcher 112 holds or forwards communications depends upon the mode in which dispatcher 112 operates. For example, in a step mode, dispatcher 112 may store some or all of the communication from client 106-1 without forwarding the communication to data sources 102 and 104. In such a mode, dispatcher 112 only forwards the communication to a data source if instructed to do so by the appropriate service 114 or if placed into stream mode by the appropriate service 114.
  • communications e.g. packets
  • dispatcher 112 directly forwards the communications to data source(s) 102 and/or 104 and services 114 separately perform their functions.
  • whether dispatcher 112 holds or forwards communications depends upon the mode in which dispatcher 112 operates. For example, in a step mode, dispatcher 112
  • dispatcher 112 does provide the communication to service 114-1, for example for client 106-1 to be authenticated and/or for other functions. If client 106-1 is authenticated, dispatcher 112 may be placed in stream mode by service 114-1. Consequently, dispatcher 112 forwards the communication to the appropriate data source(s) 102. Because dispatcher 112 is now in stream mode, subsequent communications from client 106-1 may then be forwarded by dispatcher 112 directly to the appropriate data source(s) 102 and/or 104, even if the subsequent communications are also provided to a service 114 for other and/or additional functions. Thus, dispatcher 112 may provide the communication to the data source(s) as received/without waiting for a response from a service 114.
  • responses from data source(s) 102 and/or 104 are also inspected by sidecar 110 and provided to clients 106 only if the responses are authorized.
  • a response from a data source may include data or other transmission from the data source to the client requesting access.
  • responses from data source(s) 102 and/or 104 may bypass sidecar 110 and be provided directly to clients 106.
  • data source 104 may bypass sidecar 110 and provide responses directly to client 106-1.
  • Services 114 provide security and other functions for data sources 102 and 104 and clients 106.
  • services 114 may include one or more of authentication, query analysis, query rewriting, caching, tokenization and/or encryption of data, caching, advanced or multifactor authentication, federated identity management, and/or other services. Further, one or more of the services described herein may be used together.
  • Services 114 perform more functions than dispatcher 112 and may be application layer (Layer 7) components. In contrast to dispatcher 112, services 114 may perform a deeper inspection of communications from clients 106 in order to provide various functions. The services 114 performing their functions may thus be decoupled from forwarding of communications to data source(s) 102 and/or 104 by dispatcher 112.
  • a client or communication is determined by a service 114 to be unauthorized or otherwise invalid, the communication may be recalled, or canceled, from data source(s) 102 and/or 104 and connection to the client terminated.
  • the communication may be recalled despite the decoupling of tasks performed by services 114 with forwarding of communications by dispatcher 112 because data sources 102 and 104 typically take significantly more time to perform tasks than services 114.
  • the time taken by data source 102 and 104 may be due to issues such as transmission over a network from sidecar 110 to data sources 102 and 104, queues at data sources 102 and 104, and/or other delays.
  • services 114 may perform authentication. For example, suppose service 114-1 validates credentials of clients 106 for data sources 102 and 104. In some such embodiments, service 114-1 may simply employ a username and password combination. In other embodiments, multifactor authentication (MFA), certificates and/or other higher level authorization is provided by one or more services 114. Such authentication is described herein. However, dispatcher 112 may still be a data agnostic component, such as a Layer 4 component.
  • MFA multifactor authentication
  • dispatcher 112 may still be a data agnostic component, such as a Layer 4 component.
  • routines or other lightweight process(s) may be facilitated by routines or other lightweight process(s).
  • a client such as client 106-2 may request access to data source 104 via a particular port.
  • Sidecar 110 may utilize listener(s) (not shown in FIG. 1) on the ports to identify requests for data sources 102 and 104.
  • a connection to the client 106-2 is established for the sidecar 110 on that port and a routine corresponding to the connection generated.
  • the routine is responsible for that connection only.
  • the communication from client 106-2 is also provided to dispatcher 112.
  • Dispatcher 112 provides the communication to the appropriate service(s) 114 for authentication, for example via a message bus (not shown in FIG. 1).
  • Dispatcher 112 may hold (in step mode) or forward (in stream mode) the communication to the data source(s) 102 and/or 104. If client 106-2 is not authenticated or is later determined by service(s) 114 to be unauthorized, then the service(s) 114 indicates this to dispatcher 112. For example, service(s) 114 may provide a message to dispatcher 112 via the message bus that client 106-2 is not authorized/that the corresponding routine has an unauthorized connection. Dispatcher 112 communicates with the corresponding routine, which terminates the connection to client 106- 2. Thus, connections to clients 106 may be securely managed using data agnostic, Layer 4 dispatcher 112.
  • data sources 102 and 104 may be secured and other features may be provided via service(s) 114.
  • sidecar 110 may function with a variety of data sources 102 and 104 that do not share a platform or are otherwise incompatible. Deployment of sidecar 110, for example either in the cloud or on premises, does not require changes in existing code. Consequently, implementation of sidecar 110 may be seamless and relatively easy for developers. Further, sidecar 110 need not protect every component within a particular organization. Instead, only selected data sources may be protected.
  • Use of services 114 for security as described herein may be both more effective at securing sensitive data and less expensive because data sources may not significantly increase in number even when the number of applications that access the data sources grows significantly. Further, utilizing services 114, the level of security and/or functions provided by sidecar 110 may differ for different data sources. Additional functionality may also be provided by services 114.
  • FIG. 2 is a diagram depicting another exemplary embodiment of a system 200 utilizing a protective layer between clients and data sources.
  • System 200 is analogous to system 100 and includes components that are labeled similarly.
  • System 200 indicates that multiple sidecars having different services may be used.
  • system 200 includes data sources 202-1, 202-2 (collectively 202) and 204, clients 206-1, 206-2 and 206-3 (collectively clients 206) and sidecars 210A and 210B (collectively sidecars 210).
  • data sources 202-1, 202-2 and 204, three clients 206 and two sidecars 210 are shown, in another embodiment, different numbers of data sources, clients, and/or sidecars may be used.
  • Data sources 202-1, 202-2 and 204 and clients 206 are analogous to data sources 102 and 104 and clients 106, respectively.
  • Sidecars 210A and 210B are analogous to sidecar 110.
  • sidecar 210A includes dispatcher 212A and services 214-1 A and 214-2 A (collectively services 214A).
  • sidecar 210B includes dispatcher 212B and services 214-1B, 214- 2B and 214-3B (collectively services 214).
  • Services 214A may differ from or be included in services 214B.
  • Sidecar 210A controls accesses to data sources 202, while sidecar 210B controls accesses to data source 204 in a manner analogous to described elsewhere herein.
  • one sidecar having multiple services may function for all the data sources in an organization. However, as depicted in FIG. 2, nothing prevents the use of multiple sidecars.
  • sidecars 210A and 210B are shown as controlling access to different data sources 202 and 204, in other embodiments, sidecars may control the same data source. For example, in another embodiment, sidecar 210B might serve both data source 202-1 and data source 204.
  • FIG. 3 is a diagram depicting another exemplary embodiment of a system 300 utilizing a protective layer between clients and data sources.
  • System 300 is analogous to systems 100 and 200 and includes components that are labeled similarly.
  • System 300 also includes collector 320-1, 320-2 and 320-3 (collectively collectors 320).
  • system 300 includes data sources 302 and 304, clients 306-1, 306-2 and 306-3 (collectively clients 306) as well as client 306-4 and sidecar 310.
  • two data sources 302 and 304, four clients 306 and one sidecar 310 are shown, in another embodiment, different numbers of data sources, clients, and/or sidecars may be used.
  • Data sources 302 and 304 and clients 306 are analogous to data sources 102 and 104 and clients 106, respectively.
  • Sidecar 310 is analogous to sidecar 110.
  • sidecar 310 includes dispatcher 212 and services 314-1, 314-2, 314-3, 314-4 and 314-5 (collectively services 314).
  • Sidecar 310 controls accesses to data sources 302 and 304.
  • utilities 330-1 and 330-2 that might be used by services 314.
  • service 314-1 might perform authentication and multifactor authentication using utility 330-1.
  • Service 314-6 may perform federated identity management using utility 330-2.
  • Other and/or additional utilities may be used in connection with system 300, as well as with system(s) 100 and/or 200.
  • Service 314-2 might perform query analysis as described herein.
  • Service 314-3 might perform behavior modeling based on inputs from collectors 320.
  • Service 314-4 may perform tokenization and/or encryption of sensitive data.
  • Service 314-5 may rewrite queries based on the analysis performed by service 314-2. Alternatively, service 314-2 might also rewrite queries. Thus, service 314-5 might perform another function such as caching.
  • Other services not described herein may also be provided. Two or more services may be used together in some embodiments
  • Collectors 320 reside on some clients 306.
  • each of the clients 306 includes a collector.
  • not all clients 306 include a collector.
  • none of clients 306 includes a collector.
  • clients 306 may include end users, applications, and/or microservices utilized by end users.
  • clients 306 may pass communications to each other prior to the communication being provided to sidecar 310. This is indicated by dotted line between client 306-2 and client 306-3.
  • Collectors 320 intercept communications from clients 306 and append onto the communication a state of the client/application issuing the communication.
  • collector 320-1 may intercept a query or method call from the application on client 306- land examine the state of the application.
  • collector 306- 1 The type of session, get/put/post/delete commands, APIs, IP address, query attributes, method calls, order of queries etc. may be detected by collector 306- 1. These represent the context of the query/communication. Collectors 320 attach this context to the query/communication from the corresponding clients 306. In the case of microservices/multiple applications passing a query before the query is sent to a data source, the collectors 320 for each of the microservice/applications 306 may apply the context from that microservice/application. For example, a query passed from client 306-2 to client 306-3 and then to sidecar can include a first context provided by collector 320-2 and a second context provided by collector 320-3.
  • clients 306 being passed a query does not include a collector, then that client simply does not attach the context from the client. For example, if a query is passed from client 306- 1 to client 306-4, then to client 306-3, a first context from collector 320-1 and a second context from collector 320-3 are attached to the query. In such embodiments, no context is attached by client 306-4 because no collector is present for client 306-4.
  • the query and context(s) are passed to sidecar 310 when data source 302 or 304 is accessed. Over multiple accesses, the contexts can be used by sidecar 310 (e.g.
  • a service such as service 314-3 to determine the behavior (sequence of states/contexts) for each application’s accesses of data source(s) 302 and/or 304.
  • a model of the behavior e.g. using a Hidden Markov Model
  • Subsequent accesses are compared to the baseline by service 314-3 to determine whether a current query/communication matches the baseline. If not, additional validation/defense mechanisms may be employed. For example, the connection may be terminated as described herein, access to data source 302 and/or 304 may be otherwise denied and/or additional forms of validation such as MFA may be utilized via services 314.
  • System 300 may provide the benefits of systems 100 and/or 200.
  • system 300 may improve security via collectors 320.
  • end-to-end visibility, from clients 306 to data sources 302 and 304, may be provided via sidecar 310.
  • performance of system 300 may be improved.
  • FIG. 4 is a flow chart depicting an exemplary embodiment of method 400 for authenticating a client for a data source.
  • Method 400 is described in the context of system 100. However, method 400 may be used in connection with other systems including but not limited to systems 200 and 300. For simplicity, certain steps of method 400 are depicted. Method 400 may include other and/or additional steps and substeps. Further, the steps of method 400 may be performed in another order including performing portions or all of some steps in parallel. Method 400 may be carried out each time a client commences a session for communication with a data source.
  • Dispatcher 112 of sidecar 110 receives a communication requesting access to one or more data sources from a client, at 402.
  • dispatcher 112 may receive a communication requesting access to data source 102 from client 106-1.
  • the communication may be received at dispatcher 112 after a connection between sidecar 110 and client 106-1 is established and a corresponding routine or other corresponding lightweight process generated.
  • the request may also include credentials for client 106-1.
  • dispatcher 112 is in step mode. At 404, therefore, dispatcher 112 provides the communication from client 106-1 to service 114-1, which performs authentication.
  • dispatcher 112 may send the payload of the communication to service 114-1 via a message bus (not separately labeled in FIG. 1). However, because dispatcher 112 is in step mode, dispatcher 112 does not also forward the communication to the requested data source 102. Further, because dispatcher 112 is a data agnostic component such as a Layer 4 component, dispatcher 112 does not perform a deeper inspection of the communication. Instead, dispatcher 112 simply holds (e.g. stores) the communication because dispatcher 112 is in step mode. If dispatcher 112 were in stream mode, dispatcher 112 would also forward the packet to the appropriate data source 102.
  • Service 114-1 performs authentication of client 106-1, at 406.
  • a certificate and/or other credentials such as a username and password may be used to perform authentication.
  • MFA (described in further detail below) may be used.
  • collectors such as collectors 320 are present in the system, the context of the communication provided by client 106-1 may be used in authentication at 406. For example, the context appended to the communication by a collector 320 may be compared to a behavior baseline modeled by system 100 from previous communications by client 106-1 to determine whether the context sufficiently matches previous behavior. Other and/or additional authentication mechanisms may be used in some embodiments.
  • the client requesting access is not authenticated, then access to the data source is prevented, at 408.
  • the routine corresponding to the connection with client 106-1 may be notified and the connection terminated. Other mechanisms for preventing access may also be used.
  • the communication held by dispatcher 112 is also discarded. In other embodiments, if dispatcher 112 had forwarded the communication to data source 102, then the communication is recalled at 408.
  • dispatcher 112 is placed in stream mode at 410. As a result, the communication being held is forwarded to the selected data source 102 at 410.
  • future communications corresponding to the authenticated connection with client 106-1 are forwarded to the selected data source 102 and appropriate service(s) 114, at 412.
  • service 114-1 may provide a message to dispatcher 112 changing dispatcher 112 from step mode to stream mode at 410. Consequently, dispatcher 112 also forwards the communication to corresponding data source 102.
  • Future communications received at dispatcher 112 from client 106-1 via the same connection may be both provided to one of the services 114 and to the selected data source 102.
  • clients 106 are allowed to request and receive data from data source 102. However, authentication may still continue. For example, behavioral baselining described herein, periodic requests to revalidate credentials or other mechanisms may be used, at 414. If client 106-1 loses its authentication, then communications from the client to the selected data source may be recalled and further access to the data source blocked, at 414. For example, the routine responsible for the connection to client 106-1 may be notified and the connection terminated. Thus, connection to clients 106 may be securely managed using dispatcher 112 that is a data agnostic component, such as a Layer 4 component.
  • data sources 102 and 104 may be secured. Because of the use of data agnostic dispatcher 112, sidecar 110 may function with a variety of data sources 102 and 104 that do not share a platform or are otherwise incompatible. Deployment of sidecar 110, for example either in the cloud or on premises, may require no change in existing code. Consequently, implementation of sidecar 110 may be seamless and relatively easy for developers. Further, sidecar 110 need not protect every component within a particular organization. Instead, only selected data sources may be protected. Use of services 114 for security as described herein may be both more effective at securing sensitive data and less expensive because data sources may not significantly increase in number even when the number of applications that access the data sources grows significantly. Further, utilizing services 114, the level of security and/or functions provided by sidecar 110 may differ for different data sources.
  • FIG. 5 is a flow chart depicting an exemplary embodiment of method 500 for performing one or more services for a client and a data source.
  • Method 500 is described in the context of system 100. However, method 500 may be used in connection with other systems including but not limited to systems 200 and 300. For simplicity, certain steps of method 500 are depicted. Method 500 may include other and/or additional steps and substeps. Further, the steps of method 500 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 500 may be considered to be operable once authentication of the client is completed and dispatcher 112 is in stream mode.
  • Dispatcher 112 of sidecar 110 receives a communication from a client, at 502.
  • dispatcher 112 may receive a communication from client 106-2 with a query for data source 104.
  • One or more services 114 are desired to be used with the communication. Therefore, dispatcher 112 provides the communication from client 106-2 to service(s) 114, at 504.
  • dispatcher 112 forwards the communication to the requested data source 104 at 504.
  • dispatcher 112 provides the relevant portions of the communication to both the desired data source(s) and service(s). Because dispatcher 112 is a data agnostic component such as a Layer 4 component, dispatcher 112 does not perform a deeper inspection of the communication. Instead, dispatcher 112 simply forwards the communication both to the desired data source(s) 102 and/or 104 and to service(s) 114 for further processing.
  • the desired functions are provided using one or more of the services 114, at
  • services 114 may be used for authentication of various types, query analysis, federated identity management, behavioral modeling, query rewriting, caching, tokenization or encryption of sensitive data and/or other processes. Services 114 may thus be Layer 7 components. However, tasks performed by services 114 are decoupled from forwarding of the communication to data sources by dispatcher 112.
  • data sources 102 and 104 may be secured and other features may be provided via service(s) 114. Because of the use of data agnostic dispatcher 112, sidecar 110 may function with a variety of data sources 102 and 104 that do not share a platform or are otherwise incompatible. Functions performed by services 114 are decoupled from forwarding of communications to the data sources by dispatcher 112. Thus, a variety of features may be provided for data sources 102 and 104 without adversely affecting performance of data sources 102 and 104. Consequently, performance of system 100 may be improved.
  • FIG. 6 is a flow chart depicting an exemplary embodiment of method 600 for performing multifactor authentication (MFA) for a client and a data source.
  • Method 600 is described in the context of system 300. However, method 600 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 600 are depicted. Method 600 may include other and/or additional steps and substeps. Further, the steps of method 600 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 600 may be considered to be used in implementing 406 and/or 506 of method 400 and/or 500. For the purposes of explanation, suppose service 314-1 provides multi-factor authentication.
  • Method 600 maybe considered to start after the MFA service 314-1 receives the communication from dispatcher 312. Further, dispatcher 312 maybe in step mode at the start of method 600. Thus, dispatcher 312 may hold the communication instead of forwarding the communication to data source(s). In other embodiments, dispatcher 312 may be in stream mode. Dispatcher 312 may, therefore, may also provide the communication to the appropriate data sources. MFA may be performed in addition to other authentication, such as certificate or user identification/password based authentication, performed by service 314-1 or another service. Although described in the context of authentication for access to a single data source, in some embodiments, method 600 may be used to authenticate client(s) for multiple data sources.
  • Service 314-1 calls a MFA utility 330-1, at 602.
  • the MFA utility 330-1 contacted at 602 may be a third party MFA such as DUO.
  • the MFA utility 330- 1 may be part of the organization to which data source(s) 302 and/or 304 belong.
  • MFA utility 330-1 performs multi-factor authentication for the requesting client, at 604. For example, suppose end user of client 306-2 has requested access to data source 304. The user identification and password may have been validated by service 314-1. At 602, the MFA utility 330-1 is called. Thus, the end user is separately contacted by MFA utility 330-1 at 604 and requested to confirm the user’s by the MFA facility.
  • the end user may be required to enter a code or respond to a prompt on a separate device.
  • service 314-1 is informed of whether the multi-factor authentication by MFA utility 330-1 is successful.
  • service 314-l receives from MFA utility 330-1 a success indication.
  • the success indication informs MFA utility 330-1 of whether or not MFA authentication was successful.
  • service 314-1 instructs dispatcher 312 to forward communications to the requested data source 304, at 606.
  • service 314-1 in response to receiving a positive success indication (i.e. that MFA authentication is successful), directs dispatcher 312 to forward communications to the requested data source 304.
  • dispatcher 312 is instructed to change from step mode to stream mode at 606. Thus, subsequent communications may be provided both to the data source 304 and one or more service(s) 314. In other embodiments, dispatcher 312 is simply allowed to continue forwarding communications to data source 304 at 606.
  • service 314-1 instructs dispatcher 312 to prevent access to the requested data source 304, at 608.
  • service 314-1 directs dispatcher 312 to prevent access to the requested data source 304.
  • dispatcher 312 may instruct the corresponding routine to terminate the connection with the requesting client 106. If the communication has already been forwarded to data source 304, then dispatcher 312 also recalls the communication. In some embodiments, dispatcher 312 may be instructed to remain in step mode and the client requested to resubmit the credentials and/or another mechanism for authentication used. In some embodiments, other action(s) may be taken in response to MA being unsuccessful.
  • MFA may be provided for data source(s) 302 and/or 304 in a data agnostic manner.
  • Certain data sources, such as databases typically do not support MFA.
  • method 600 may provide additional security to such data sources without requiring changes to the code of data sources 302 and 304. Security of system 100 may thus be improved in a simple, cost effective manner.
  • FIG. 7 is a flow chart depicting an exemplary embodiment of method 700 for performing federated identity management for a client for a data source.
  • Federated identity management allows end users to access various facilities in an organization, such as multiple databases, email, analytics or other applications, based on a group identity and using a single set of credentials.
  • an end user may be a data analyst in a finance department. The end user may thus be considered a member of three groups: employees, data analysts and the finance department.
  • a user identification and password for the end user may allow the end user to access their company/employee email, applications for the finance department, databases including information used by the finance department such as financial projections for the organization, analytics applications accessible by data analysts and other data based on the end user’s membership in various groups within the organization.
  • Federated identity management may use protocols such as lightweight directory access protocols (LDAP) and directories defining the groups to which each end user belongs.
  • LDAP lightweight directory access protocols
  • Method 700 is described in the context of system 300. Flowever, method 700 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 700 are depicted. Method 700 may include other and/or additional steps and substeps. Further, the steps of method 700 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 700 may be considered to be used in implementing 506 of method 500. For the purposes of explanation, service 314-6 is considered to provide federated identity management. Method 700 may be considered to start after service 314-6 receives the communication from dispatcher 312.
  • Service 314-6 receives the end user’s credentials, at 702. For example, dispatcher 312 forwards to service 314-6 a communication requesting access to data source 302.
  • the communication may include the end user’s user identification and password for federated identity management.
  • the end user credentials are otherwise associated with the communication but are provided to service 314-6.
  • Service 314-6 authenticates the end user with a federated identity management utility or database 330-2, such as an LDAP directory, at 704. To authenticate the end user the user identification and password are utilized.
  • Service 314-6 searches the federated identity management database 330-2 for the group(s) to which the end user belongs, at 706.
  • sidecar 310 logs onto the data source 302 as a proxy for the end user, at 708.
  • the end user may then access data source 302 in accordance with the privilege and limitations of the group(s) to which the end user belongs.
  • federated identity management can be achieved for data source(s) 302 and/or 304.
  • Some databases do not support federated identity management.
  • Method 700 and sidecar 310 having data agnostic dispatcher 312 may allow for federated identity management for such databases without changes to the databases.
  • an end user may be able to access the desired data sources.
  • the organization can manage access to the data sources using groups in the federated identity management database. This may be achieved without requiring changes to data sources 302 and 304. Because sidecar 310 accesses data sources 302 and/or 304 as a proxy for the end user, sidecar 310 may log activities of the end user.
  • FIG. 8 is a flow chart depicting an exemplary embodiment of method 800 for performing federated identity management for a client for a data source using an LDAP directory.
  • Method 800 is described in the context of system 300. However, method 800 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 800 are depicted.
  • Method 800 may include other and/or additional steps and substeps. Further, the steps of method 800 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 800 may be considered to be used in implementing 506 of method 500 and/or 704, 706 and/or 708 of method 700.
  • service 314-6 is considered to provide federated identity management via LDAP.
  • Method 800 is considered to commence after sidecar 310 is provided with a specialized account for LDAP directory 330-2. The specialized account allows sidecar 310 to obtain information from LDAP directory 330-2 that is not available to a typical end user, such as the identification of end users and the groups to which end users belong. In some embodiments, the account is a read only account for sidecar 310.
  • Service 314-6 binds to the LDAP directory using the read only account at 802.
  • a communication requesting access to data source(s) 302 and/or 304 is received at dispatcher 310 and provided to service 314-6 in a manner analogous to 502 and 504 of method 500.
  • the communication includes the end user’s LDAP credentials.
  • the end user’s LDAP credentials are received at service 314-6.
  • service 314-6 may search for the end user in the LDAP directory using the read only account, at 804. Searching LDAP directory 330-2 allows service 314-6 to determine whether the user exists in LDAP directory 330-2. If not, sidecar 310 may prevent access to the desired data source(s). If, however, the end user is found at 804, then service 314-6 binds to the LDAP directory as a proxy for the end user, at 806.
  • Service 314-6 may then request a search for the groups to which the end user belongs, at 808. This is facilitated by the read only account for sidecar 310. Thus, service 314-6 may determine the groups to which the end user belongs as well as the privileges and limitations on each group. A group to be used for accessing the data source(s) 302 and/or 304 is selected at 810. In some embodiments, service 314-6 ranks groups based upon their privileges. A group having more privileges (e.g. able to access more data sources or more information on a particular data source) is ranked higher. In some embodiments, service 314- 6 selects the highest ranked group for the end user. In some embodiments, service 314-6 selects the lowest ranked group. In some embodiments, the user is allowed to select the group. In other embodiments, another selection mechanism may be used.
  • the desired data source(s) are accessed using the selected group, at 812.
  • the end user may access data and/or applications based upon their membership in the selected group.
  • Information related to the end user’s activities is logged by sidecar 310, at 814.
  • services 314-6 may directly log the end user activities or may utilize another service, such as query analysis, to do so.
  • an end user may be able to access the desired data sources via federated identity management performed through an LDAP directory.
  • the benefits of federated identity management may thus be achieved.
  • the end user’s actions may be logged. Thus, visibility into the activities of individual end users may be obtained.
  • FIG. 9 is a flow chart depicting an exemplary embodiment of method 900 for analyzing and logging information related to queries of a data source.
  • Method 900 is described in the context of system 100. Flowever, method 900 may be used in connection with other systems including but not limited to systems 200 and 300. For simplicity, certain steps of method 900 are depicted. Method 900 may include other and/or additional steps and substeps. Further, the steps of method 900 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 900 may be considered to be used in implementing 506 of method 500. For the purposes of explanation of method 900, service 114-1 is considered to provide query analysis and logging.
  • a client such as client 106-1 may be considered to be authenticated for data source(s) 102 and/or 104 and to perform a query for data on one or both of data sources 102 and 104.
  • the query may be an SQL query.
  • Sidecar 110 receives an identification of information of interest in the data source(s) 102 and/or 104, at 902. Also at 902, policies related to the sensitive information are also received. Reception of this information at 902 may be decoupled from receiving queries and analyzing queries for the remainder of method 900. For example, owner(s) of data source(s) 102 and/or 104 may indicated to sidecar 110 which tables, columns/rows in the tables, and/or entries in the tables include information that is of interest or sensitive. For example, tables including customer names, social security numbers (SSNs) and/or credit card numbers (CCNs) may be identified at 902.
  • SSNs social security numbers
  • CCNs credit card numbers
  • Columns within the tables indicating the SSN, CCN and customer name, and/or individual entries such as a particular customer’s name, may also be identified at 902. This identification provides to sidecar 110 information which is desired to be logged and/or otherwise managed. Further, policies related to this information are provided at 902. Whether any logging is to be performed or limited is provided to sidecar at 902. For example, any user access of customer tables may be desired to be logged. The policies indicate that queries including such accesses are to be logged. Whether data such as SSNs generated by a query of the customer table should be redacted for the log may also be indicated in the policies.
  • Sidecar 110 receives a query from a client at dispatcher 112 and provides the query to service 114-1, at 903.
  • the query may also be sent from dispatcher 112 to the appropriate data source(s) as part of 903.
  • Process 903 is analogous to 502 and 504 of method 500.
  • the query is received at service 114-1.
  • Service 114-1 parses a query provided by a client 106, at 904.
  • a client 106-1 may provide a query for data source 102 to sidecar 110.
  • Dispatcher 112 receives the query and provides the query both to data source 102 and to service 114-1.
  • Service 114-1 parses the query to determine which operations are requested and on what portions of data source 102.
  • the Service 114-1 thus emits a logical structure describing the query and based on the parsing, at 906.
  • the logical structure is an abstract syntax tree corresponding to the query.
  • Each node in the tree may represent a table being searched, operation in the query, as well as information about the operation.
  • a node may indicate a join operation or a search operation and be annotated with limitations on the operation.
  • the query is logged, at 908.
  • the log may include the end user/client 106-1 that provided the query as well as the query string.
  • the features extracted from the abstract syntax tree may be logged in a manner that is indexable or otherwise more accessible to analytics.
  • the log may be configured to be human readable.
  • a JSON log may be used. For example, a list of the operations and tables accessed in the query may be included in the log. Sensitive information such as SSN may be redacted from the log in accordance with the identification of sensitive information and policies relating to sensitive information received at 902. Thus, a placeholder may be provided in the log in lieu of the actual sensitive information accessed by the query.
  • the logical structure and/or log are analyzed at 909. This process may include analyzing the abstract syntax tree and/or information in the log.
  • additional action may be taken by sidecar 110, at 910.
  • a query rewriting service that is part of service 114-1 or a separate service may be employed if it is determined in 909 that the log generated in 908 indicates that the query may adversely affect performance. For example, limits may be placed on a query, clauses such as an “OR” clause and/or a tautology identified and/or removed. As a result, queries that result in too many rows being returned may be rewritten to reduce the number of rows. If the log or other portion of the query analysis indicates that the query may represent an attack, then access to the data source may be denied at 910.
  • the analysis at 909 of the logical structure and log may indicate that the query includes wildcards or tautologies in users’ names.
  • the corresponding routine may terminate the connection to the client from which the query originated. If the query has been passed on to data source 102, then the query may be canceled at 910. Unwanted exfiltration of sensitive information may thus be prevented.
  • the query analysis indicates that a similar query was recently serviced, then some or all of the information for the similar query that already exists in a cache may be used to service the query. If the query can be completely serviced by information in the cache, then the query may be recalled from/canceled before or during servicing by data source 102.
  • various actions may be taken based upon the analysis of the query by service 114-1.
  • data source 102 includes a customer table of customer information having columns of customer names, customer SSNs, customer CCNs, tokenized CCNs (e.g. CCN encrypted with FPE or represented by a token), and customer identifiers (CIDs).
  • data source 102 also includes an order table including a table of customer orders.
  • the table includes a column of order customer identifiers (OCIDs) and multiple columns of orders for each customer identifier. In each order column, the item prices for the order are indicated.
  • the order customer identifier for the order table is the same as the customer identifier in the customer table for data source 102.
  • Query analysis and logging may be performed by service 114-1.
  • service 114-1 is informed that the customer table and the columns of customer names, customer SSNs and (tokenized) customer CCNs are sensitive information for which activity is desired to be logged. Also at 902, service 114-1 is informed that customer names and SSNs are to be redacted from the log.
  • FIG. 10 depicts the corresponding abstract syntax tree 1000 generated from the query at 906.
  • the abstract syntax tree has been annotated for clarity. Nodes 1002, 1004, 1012, 1022 and 1032 and lines connecting nodes 1002, 1004, 1012, 1022 and 1032 represent the query.
  • the string forming the query may also be provided in the log. However, because they were not identified as being of interest, the order table, CID, OCID and object price are not included in the indexable portion of the log.
  • Method 900 may facilitate analysis of queries performed, aid in response to attacks, and/or improve performance of the data source. Because dispatcher 110 is data agnostic and may be a transport layer component, this may be achieved without requiring changes to data sources 102 and 104 while maintaining stability of the data sources 102 and 104. Thus, performance and security for system 100 may be enhanced.
  • FIGS. 11 A and 1 IB are flow charts depicting exemplary embodiments of methods for utilizing tokenization and/or encryption of sensitive data.
  • FIG. 11 A is a flow chart depicting an exemplary embodiment of method 1100 for using tokenization and/or encryption for storing data at a data source.
  • Method 1100 is described in the context of system 300. However, method 1100 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 1100 are depicted. Method 1100 may include other and/or additional steps and substeps. Further, the steps of method 1100 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 1100 may be considered to be used in implementing 506 of method 500.
  • Method 1100 may be considered to start after system 300 receives policies indicating how sensitive data are to be treated. For example, policies indicating what data are sensitive (e.g. which tables/entries include sensitive data), what clients are allowed to have access to the sensitive data, for what purposes client(s) are allowed to have access to the sensitive data, how the sensitive data are to be anonymized (e.g. tokenized and/or encrypted), and/or other information desired by controller of data sources 302 and/or 304 have already been received by sidecar 310 and provided to the appropriate service(s). Although described in the context of access to a single data source, in some embodiments, method 1100 may be used for multiple data sources. In some embodiments, the same service fulfills request to store sensitive data and requests to obtain sensitive data.
  • policies indicating what data are sensitive e.g. which tables/entries include sensitive data
  • policies indicating what clients are allowed to have access to the sensitive data e.g. which tables/entries include sensitive data
  • what purposes client(s) are allowed to have access to the sensitive data
  • some service(s) may service requests to store data/tokenize data while other service(s) are used obtain the tokenized data. However, such services communicate in order to service at least some of the requests.
  • the same service may utilize different types of anonymization (e.g. tokenization and encryption).
  • different services may be used for different types of anonymization. For example, one service may tokenize data while another service encrypts data.
  • Method 1100 is described as being used in connection with method 1150. In other embodiments, method 1100 may be used with a different method for accessing encrypted/tokenized data.
  • a request from a client to store sensitive data at a data source is received by a sidecar, at 1102.
  • the dispatcher which is data agnostic, forwards the request to an encryption/tokenization service for anonymization of the sensitive data desired to be stored, at 1104.
  • the sensitive data is and anonymized, at 1106.
  • the data desired to be stored includes sensitive data desired to be anonymized as well as data that need not by anonymized.
  • 1106 also includes identifying the sensitive data to be anonymized.
  • anonymizing data includes encrypting and/or tokenizing the data.
  • encryption such as format preserving encryption (FPE) may be used.
  • CCNs and SSNs may be encrypted using FPE such that the encrypted data has the same number of digits as the CCN and SSN (i.e. such that the format is preserved) but does not have intrinsic meaning.
  • the alphanumeric string having nine members may replace an SSN.
  • Other types of encryption, tokenization, and/or data masking may also be used at 1106.
  • the sensitive data is anonymized. Because policies may be used to determine how and what data are encrypted/tokenized, 1106 is performed on an attribute level. For example, the CCN of a user may be encrypted by FPE, but the SSN of the same user may be replaced by a token based on the policies used by the encryption/tokenization service.
  • the anonymized data is stored in the data source, at 1108.
  • the anonymized data may be retained in place of the actual sensitive data.
  • the sensitive data may also be stored, for example in a secure data vault, which may require enhanced authentication to access.
  • sensitive data may be tokenized and/or encrypted and stored using a data agnostic dispatcher.
  • FIG. 1 IB is a flow chart depicting an exemplary embodiment of method 1150 for accessing tokenized and/or encrypted data from a data source.
  • Method 1150 is described in the context of system 300. However, method 1150 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 1150 are depicted. Method 1150 may include other and/or additional steps and substeps. Further, the steps of method 1150 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 1150 may be considered to be used in implementing 506 of method 500. Method 1150 may be considered to start after system 300 receives policies indicating how sensitive data are to be treated. For example, policies indicating what data are sensitive (e.g.
  • method 1150 may be used for multiple data sources.
  • the same service fulfills request to store sensitive data and requests to obtain sensitive data.
  • some service(s) may service requests to store data/tokenize data while other service(s) are used obtain the tokenized data.
  • Such services communicate in order to service at least some of the requests.
  • the same service may utilize different types of anonymization (e.g. tokenization and encryption).
  • different services may be used for different types of anonymization.
  • one service may tokenize data while another service encrypts data.
  • Method 1150 is described as being used in connection with method 1100. In other embodiments, method 1150 may be used with a different method for anonymizing data.
  • a request for the sensitive data stored at data source is received by the sidecar, at 1152.
  • the request may come from the same client that stored the data or a different client. Because request(s) for data may be independent of storage, 1152 through 1162 may be decoupled from 1102 through 1108. For example, the request may be received at 1152 at a different time, or may not be received. Thus, methods 1100 and 1150 are separately described.
  • the dispatcher provides the request to access sensitive data to encryption/tokenization service, at 1154. The request may also be forwarded to the data source storing the anonymized data.
  • the encryption/tokenization service determines what type of authorization the requestor possesses, at 1156.
  • the requester may only be authorized to receive the anonymized (e.g. tokenized/encrypted) data.
  • the requesting client might be a computer system of data scientist associated with system 300.
  • the data scientist/client may be allowed to track use of a credit card number, but not be authorized to know the actual credit card number.
  • the requester may be authorized to receive the original, sensitive data.
  • the requesting client might be a merchant’s payment system or the original user’s computer systems, both of which may be authorized to receive the de-anonymized (e.g. unencrypted/de-tokenized) sensitive data.
  • the requester may be unauthorized to receive either data.
  • the requesting client might be a malicious individual attempting to steal the sensitive data.
  • the encryption/tokenization service validates credentials for the requesting client.
  • the encryption/tokenization service may use passwords, certificates, multifactor authentication, behavioral baselining through collectors) and or other mechanism(s).
  • encryption tokenization service may call another service to perform authentication at 1156.
  • the anonymized data stored at the data source is retrieved, de-anonymized and provided to client, at 1158.
  • encryption tokenization service may decrypt and or detokenize the data that was stored in the data source.
  • encryption/tokenization service may retrieve the original, sensitive data from a secure data vault (not shown in FIGS. 3 and 11 A-l IB). The sensitive data is then sent to the authorized requester at 1158.
  • the requesting client is determined to be authorized to receive only the anonymized data, then this anonymized data are retrieved and sent to the requester, at 1160.
  • encryption/tokenization service may simply retrieve the anonymized data from the data source and forward this data to the requesting client.
  • a requester may be authorized to receive either or both of the sensitive data and the anonymized data.
  • 1158 and/or 1160 may include determining whether the requester has selected the anonymized/de-anonymized data and providing the anonymized/de-anonymized data.
  • both the anonymized and the de anonymized data might be provided.
  • the routine may terminate the connection to client as described above, the communication may be recalled from the data source, the client may be blacklisted, managers of system 300 and/or owner of the sensitive data may be notified of the attempted breach and/or other action taken.
  • the corresponding routine may terminate the connection to the client from which the query originated. If the query has been passed on to the data source, then the query may be canceled at 1162. Unwanted exfiltration of sensitive information may thus be prevented.
  • step 1106 might be skipped and the sensitive data stored at 1108.
  • step 1106 might be skipped and the sensitive data stored at 1108.
  • no decryption is performed for the requester determined to be authorized to receive the sensitive data.
  • the data are encrypted/tokenized and then provided at 1160.
  • methods 1100 and 1150 may be adapted to the case where sensitive data are stored.
  • Dispatcher 312 forwards the request to encryption/tokenization service 314-2 for anonymization, at 1104. Based on the policies provided and/or capabilities of encryption/tokenization service 314-2, the sensitive data is identified and anonymized, at 1106. For example, encryption/tokenization service 314-2 may encrypt some sensitive data and tokenize other sensitive data. The anonymized data is stored in data source 302, at 1108. [0096] A request from client 306-2 for the sensitive data stored at the data source is received by the sidecar 310, at 1152. Dispatcher 310 provides the request to access sensitive data to encryption/tokenization service 314-2, at 112. The request may also be forwarded by dispatcher 312 to data source 302.
  • Encryption/tokenization service 314-2 determines what type of authorization the requestor possesses, at 1156. Thus, encryption/tokenization service 314-2 validates credentials for the requesting client 306-2.
  • encryption/tokenization service 314-2 may retrieve the original, sensitive data from a secure data vault. The sensitive data is then sent to the authorized requester. If the requesting client 306-2 is determined to be authorized to receive only the anonymized data, then encryption/tokenization service 314-2 retrieves the anonymized data from data source 302 and forwards this data to the requesting client 306-2.
  • routine may terminate the connection to client 306-2, the communication may be canceled or recalled from data source 302, client 306-2 may be blacklisted, managers of system 300 and/or owner of the sensitive data (e.g. user of client 306-1) may be notified of the attempted breach and/or other action taken.
  • Using methods 1100 and 1150 sensitive data may be more securely stored and retrieved. Instead of storing sensitive data, anonymized data may be stored at 1108. How and what data are anonymized may be determined on an attribute level, which improves flexibility of methods 1100 and 1150. This improves the ability of system 300 and methods 1100 and 1150 to protect sensitive data from being inappropriately accessed. Because these functions are provided via service(s) 314, the enhanced security may be provided for data source(s) 302 and/or 304 that do not otherwise support encrypted data. Stated differently, secure storage and encryption/tokenization of data may be performed in a data agnostic manner. Thus, methods 1100 and 1150 may provide additional security to such data sources without requiring changes to the code of data sources 302 and 304.
  • FIGS. 12A and 12B are flow charts depicting exemplary embodiments of methods for providing client information and for performing behavioral baselining for clients.
  • FIG. 12A is a flow chart depicting an exemplary embodiment of method 1200 for providing client information and may be used as part of performing behavioral baselining for a client.
  • Method 1200 is described in the context of system 300. However, method 1200 may be used in connection with other systems including but not limited to systems 100 and 200 that employ collectors such as collectors 320. For simplicity, certain steps of method 1200 are depicted. Method 1200 may include other and/or additional steps and substeps. Further, the steps of method 1200 may be performed in another order including performing portions or all of some steps in parallel.
  • method 1200 may be considered to be used in implementing 506 of method 500.
  • Method 1200 is described in the context of clients 306-2 and 306-3, collectors 320-2 and 320-3, service 314-2 and data source 302. Thus, method 1200 commences after collectors 320 have been provided on one or more clients 306 utilizing data sources. However, in other embodiments, other clients, collectors, service(s) and/or other data sources may be used.
  • Communications for data source(s) to be issued by a client are intercepted, for example by a collector at the client, at 1202.
  • queries, method or API calls, commands or other messages may be intercepted before being provided from the client for transmission to the sidecar.
  • a collector may attach itself to a client application and use Java Database Connectivity (JDBC) to intercept queries from the client of the data source(s).
  • JDBC Java Database Connectivity
  • the state of the client issuing the communication is determined and attached to/associated with the intercepted communication, at 1204.
  • the type of call e.g., the type of call, the type of session/session identification, user identification for the session, the type of command (e.g. get, put, post, and delete commands), APIs, IP address, query attributes, method calls, order of queries, and/or application making the calls may be detected by the collector and attached to the communication at 1204.
  • These attributes represent the context, or state, of the client (or client application) when issuing the communication.
  • the collector attaches this context/state to the query or other communication being provided from the client.
  • the communication and attached state are sent from the client, at 1206. In some embodiments, the attached state may be considered to be part of or included in the communication sent from the client.
  • other clients may receive the communication from the sending client, perform other functions and then issue another communication.
  • multiple clients may send and receive a communication before the communication is provided to the sidecar or data source.
  • any outgoing communication is intercepted as in 1202, the context for that client is determined and attached to the communication as in 1204 and the communication and state/context sent as in 1206, via 1208. If only a single client having a collector sends the communication to the sidecar, then 1208 may be omitted. If five clients having collectors send the communication in series, then the originating client performs 1202, 1204 and 1206. 1208 may be repeated four times for the four additional clients receiving and sending the communication.
  • each of the clients having a collector can attach their state to the communication. Further, the states may be attached in the order in which the clients sent/received the communication. The last client sending the communication provides the communication to a sidecar, such as sidecar 310.
  • the context for a client can be provided to along with the communication.
  • the series of contexts provided with these communications may represent typical behavior for the client during interaction with the data source.
  • the client(s) may send information relating to their state and/or behavior in addition to communications such as queries.
  • FIG. 12B is a flow chart depicting an exemplary embodiment of method 1250 for performing behavioral baselining for a client.
  • Method 1250 is described in the context of system 300. However, method 1250 may be used in connection with other systems including but not limited to systems 100 and 200 that employ collectors such as collectors 320. For simplicity, certain steps of method 1250 are depicted. Method 1250 may include other and/or additional steps and substeps. Further, the steps of method 1250 may be performed in another order including performing portions or all of some steps in parallel. In some embodiments, method 1250 may be considered to be used in implementing 506 of method 500. Method 1250 is described in the context of clients 306-2 and 306-3, collectors 320-2 and 320-3, service 314-2 and data source 302.
  • method 1250 commences after collectors 320 have been provided on one or more clients 306 utilizing data sources. However, in other embodiments, other clients, collectors, service(s) and/or other data sources may be used. Method 1250 may be performed in conjunction with method 1200 and so may receive communications and states/contexts provided via method 1200.
  • the communication and context/s) of the client(s) are received at the sidecar, at 1252.
  • the sidecar thus receives the communication, which may include multiple queries or method calls, as well as the states of all clients having collectors which sent the communication along before reaching the sidecar.
  • the communication and attached context(s) are received at the dispatcher.
  • the communication and attached context sent by the client at 1206 or 1208 of method 1200 is received at the sidecar at 1252.
  • the context/s) are forwarded from the dispatcher to behavioral baselining service(s), at 1254.
  • the communications with which the context/s) are associated are also provided to the behavioral baselining service(s) at 1254.
  • the dispatcher may send the communication on to the desired data source(s).
  • processing of the query or other calls in the communication may not be delayed by inspection of the context/s) of clients and other functions performed by behavioral baselining service(s).
  • the communication may be held at the dispatcher until behavioral baselining is completed. This may occur, for example, if the dispatcher is in step mode described above.
  • the state(s)/context(s) for the client(s) associated with the communication are compared with baseline(s) for client(s), at 1256.
  • the communication is also part of this comparison.
  • the particular query of the database provided by the client as well as the state of the client may be used for comparison with the baseline.
  • just the context(s) might be used.
  • a single context of a client associated with a single communication is compared to the baseline(s) at 1256.
  • multiple contexts that may be in a particular order of a client are compared to the baseline at 1256.
  • the behavioral baselining service may store the context received for each communication for each client having a collector.
  • a client issues multiple communications for a data source when utilizing the data source.
  • a set of these contexts for a particular client represents the behavior of that client around the time the client interacts with the data source.
  • the behavioral baselining service analyzes the behavior (series of contexts) of the client(s) providing the communication(s). In some embodiments, only the identities of the contexts are used. In some embodiments, the identities of the contexts as well as their order are used for comparison. In some embodiments, the behavioral baselining service compares the context(s) to the behavior based upon a model of the behavior (the series of states/contexts), such as a Hidden Markov Model.
  • the behavioral baselining service maintains a model of requesting client(s)’ behavior and compares the context in the current communication to the behavior.
  • a single context may be compared to the baseline in some cases and behavior in others. For example, for a first communication received by the sidecar, that first communication may be compared to the baseline. As additional communications are received, these communications may be compared to the baseline at 1256.
  • a client might first be authenticated and granted access to a data source based on another method of authentication, such as MFA. Once the client sends additional communication(s) with additional context(s), these communication(s) and contexts) may be used to compare the behavior for the client with the baseline.
  • the initial communication and authentication may be considered part of the behavior. In other embodiments, the initial communication and authentication may be considered separately from subsequent communication(s) and state(s).
  • the requesting client(s) are allowed access to the data source, at 1258.
  • the data source is allowed to service the communication(s) provided by the client(s).
  • the desired action is taken, at 1260.
  • the action taken may depend upon the mismatch determined in 1256 or on other factors. For example, the client(s) initiating the communication(s) may not be allowed to access the data source. In such cases, the dispatcher may be informed and the corresponding routine used to terminate the connection to client(s). If the communication had already been forwarded to the data source(s), then the communication may be recalled from the data source(s).
  • the authentication may be revoked. In such embodiments, the dispatcher may be informed the client is unauthorized and the corresponding routine used to terminate the connection to client(s). Communication(s) that had been forwarded to the data source(s) may also be recalled from the data source(s). If the mismatch is sufficiently great or occurs greater than a threshold number of times, or at least a particular number of times in a row, then the client(s) may be blacklisted. In some embodiments, a secondary mechanism of authentication, such as MFA, may be invoked at 1260. Thus, access to the data source(s) may be determined at least in part based upon behavior of the requesting client(s). These and/or other actions may be taken at 1260.
  • MFA secondary mechanism of authentication
  • the model/baseline may be updated, at 1262. For example, if it is determined that the context sufficiently matches the behavior at 1258, then the model/baseline may be updated with the context in the communication from client(s). If the context is considered inconsistent with the baseline, then the model/baseline may be updated with this information.
  • collector 320-2 in client 306-2 intercepts a communication including a query of data source 302 at 1202.
  • the context of client 306-2 is determined by collector 320-2 and attached to the query.
  • Client 306-2 then provides the communication and context to sidecar 310. Because client 306-2 provides the communication to sidecar 310 without providing the communication to another client 306, 1208 is skipped.
  • Dispatcher 312 receives the communication at 1252 and provides the communication and context to behavioral baselining service 314-2 at 1254.
  • the communication is also passed to data source 302 at 1254.
  • Behavioral baselining service 314-2 compares the context received at 1254 to the baseline for client 306-2 at 1256. If the context received is consistent with the baseline, then access is allowed to data source 302, at 1258. Otherwise, access may be denied, for example the connection terminated, at 1260. Additional actions may also be taken at 1260 such as blacklisting client 306-2.
  • the baseline may also be updated at 1262.
  • multiple applications in multiple clients may pass a communication before the communication is sent to a data source. For example, this may occur where microservices are employed, as discussed above.
  • collector 320-2 in client 306-2 intercepts the communication including a query of data source 302 at 1202.
  • the state of client 306-2 is determined by collector 320-2 and attached to the query.
  • Client 306-2 then provides the communication and state to client 306-3.
  • client 306-3 may add another query to the communication or otherwise modify the communication.
  • Collector 320-3 in client 306-3 intercepts the communication, attaches the state of client 306- 3 and provides the communication to sidecar 310 at 1208.
  • the communication now includes the states of clients 306-2 and 306-3.
  • Dispatcher 312 receives the communication at 1252 and provides the communication and states to behavioral baselining service 314-2 at 1254. The communication is also passed to data source 302 at 1254. Behavioral baselining service 314-2 compares the states received at 1254 to the baselines for clients 306-2 and 306-3 at 1256. If the states received are consistent with the baselines, then access is allowed to data source 302, at 1258. Otherwise, access may be denied, for example the connection terminated and the communication recalled from data source 302, at 1260. Additional actions may also be taken at 1260 such as blacklisting client 306-2 and/or 306-3. The baseline(s) may also be updated at 1262.
  • methods 1200 and 1250 security and performance for data sources may be improved.
  • the context(s)/state(s) of client(s) in communications requesting access to data source(s) may be analyzed to determine whether the communication is consistent with previous behavior of client(s). If the state(s) of the client(s) are inconsistent with the baseline, then access to the data source(s) may be prevented and/or additional action taken.
  • Methods 1200 and 1250 may also be extended to compare behavior (a series of states, for example for multiple queries) of clients to previous behavior and authenticate clients based upon their behavior. Thus, attacks from a client that has been hijacked may be detected and addressed. Further, collectors need not be present on all clients to enhance security.
  • Methods 1200 and/or 1250 may be coupled with other methods, such as query analysis in method 900, authentication using method 400, tokenization in method 1100 and/or MFA in method 600 to further improve security.
  • sidecar 310 may provide multifactor authentication and/or federated identity management via service 314-1, query analysis via service 314-2, query rewriting via service 314-3 (or as part of service 314-2), behavioral baselining via service 314-4, tokenization of sensitive data via service 314-5, and/or other service other functionality via service 314-6.
  • dispatcher 312 may forward some or all of the request to multiple services 314 and/or services 314 may communicate with other services in order to perform the desired functions.
  • performance of and security for data sources 302 and/or 304 may be improved by a variety of services without altering data sources 302 and/or 304.
  • new services may be added to sidecar 310 for data source(s) 302 and/or 304 by adding service module or updating services 314. These new services may be added without altering the code of data sources 302 and 304. Thus, performance, security and/or other features of system 300 can continue to be improved.
  • techniques and mechanisms described herein can be used for intruder detection.
  • Communications for a data source in an organization are intercepted and analyzed to identify an intruder detection signature.
  • all communications for the data source are intercepted.
  • all intercepted communications are analyzed.
  • a subset of intercepted communications may be analyzed.
  • An intrusion is determined based on the intruder detection signature and an alarm generated based on the intrusion.
  • the dispatcher e.g. an OSI Layer 4 component
  • the appropriate service(s) e.g. OSI Layer 7 component(s) analyze the communications, determine whether an intrusion has occurred and determine the alarm.
  • the intruder detection signature includes at least one of a port scanning pattern, a credential guessing pattern, a disallowed exfiltration pattern, a password spraying pattern, and a trickle exfiltration pattern.
  • the communications are analyzed to determine whether a behavioral baseline for the data source is matched.
  • the behavioral baseline is also dynamically updated using the communications. Communication(s) are identified as matching the intruder detection signature in response to the communication(s) failing to match the behavioral baseline.
  • the communications may include a request to send data to a port for the organization.
  • determining whether communication(s) match the behavioral baseline includes determining at least one of: whether the data matches a client signature for the port, whether the request is from a client having unsuccessful login attempts (e.g. a credential guessing pattern), and whether the client requests unauthorized secured data (e.g. a disallowed exfiltration pattern, SQL injection attacks).
  • port scanning may be identified and an alarm provided.
  • analyzing includes determining whether a portion of the communications from client(s) include repeated (e.g. periodic) unsuccessful data source access attempts for a multiple data source access accounts.
  • communication(s) are identified as matching the intruder detection signature in response to a determination that the communication(s) include the repeated unsuccessful data source access attempts.
  • password spraying may be identified and an alarm provided.
  • analyzing includes determining whether request(s) for data for a particular data set are received from a client in a time interval and whether requests from the client match a work signature. Requests are identified as matching the intruder detection signature in response to the requests for the particular data set being determined to have been received in the time interval in combination with the requests being determined to fail to match the work signature. Thus, in some embodiments, trickle exfiltration may be identified and an alarm provided. Consequently, various intrusions for a network or organization may be detected and a response provided.
  • FIG. 13 is a flow chart depicting an embodiment of method 1300 for detecting intrusions, or threats, to a network.
  • Method 1300 is described in the context of system 300 (which, in some embodiments, may exclude collectors 320 for method 1300). However, method 1300 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 1300 are depicted. Method 1300 may include other and/or additional steps and substeps. Further, the steps of method 1300 may be performed in another order, including performing portions or all of some steps in parallel. In some embodiments, method 1300 may be considered to be used in implementing 506 of method 500. Method 1300 may be considered to start after the receipt of policies indicating how intrusions are to be treated.
  • alarms for various intrusions, models used in detecting signatures are and/or other information desired by controller of the data sources have already been received by sidecar and provided to the appropriate service(s).
  • alarms for intrusions may include one or more of a connection being terminated, a client or user being temporarily or permanently blacklisted (prevented from accessing data sources and/or other portions of the network), the exfiltration of data prevented, queries suspended or terminated, a notification being provided to an administrator of the network and/or other actions being taken.
  • method 1300 may be used for multiple data sources.
  • a single service may perform method 1300.
  • multiple services may be utilized in performing method 1300.
  • some service(s) analyze communications while other service(s) are used to provide alarms. Further, service(s) may also employ utilities, such as utilities 330 in performing method 1300.
  • Communications for data source(s) in an organization are intercepted, at 1302. In some embodiments, all communications are intercepted at 1302. In some embodiments, all communications to the data source(s) are intercepted.
  • requests to send data to a port that is part of a network for the organization and which may be coupled with the data source(s), requests to store data at data source(s), requests to receive data from data source(s), requests for authentication to access data source(s) and/or other portions of the network, requests to modify portions of the network, requests to alter features of user accounts such as passwords, and/or other communications for data sources in the network for the organization are intercepted at 1302.
  • communications from the data source are intercepted. For example, data provided in response to a request, notifications of invalid attempts to log into the network and/or data source, notifications of successful authentications, and/or other communications from the data source are intercepted at 1302.
  • all of the communications from the data source may be intercepted. In some embodiments, only some of the communications from the data source may be intercepted. In some embodiments, none of the communications from the data source may be intercepted. Thus, all communications to and/or from data source(s) are intercepted at 1302.
  • the communications that are intercepted are analyzed to identify intruder detection signature(s), at 1304.
  • 1304 may include extracting features of the communication and providing features of the communication to a model, such as a hidden Markov model, for analysis.
  • the model may be utilized to identify patterns in the communications such as types of data sent to or received from ports and/or data sources; frequency of particular communications such as repeated invalid login attempts or repeated requests for data which a recipient is not authorized to receive; repeated types of communications such as unsuccessful data source access attempts across multiple accounts or multiple accesses of the same data set to scan through the data set; a periodic nature of communications such as attempts to access multiple user accounts over a longer period of time (e.g. days, weeks or months); SQL injection attacks (e.g.
  • the analysis at 1304 may also include statistical analysis of the communications. Such statistical analysis may uncover patterns corresponding to intruder detection signatures. For example, clustering of particular types of communications (e.g. k- means clustering or another clustering technique) and/or identification of the type of distribution (e.g. Gaussian distribution) for communications may be employed. Based upon the statistical analysis, a particular pattern may emerge. For example, invalid login attempts may be clustered around a particular time of day and/or around a particular frequency (e.g. weekly).
  • communications are compared to domain specific knowledge at 1304. Domain specific knowledge includes information about the types, numbers and order of queries that authorized users typically employ.
  • authorized users generally do not simply repeatedly request sensitive data. Instead, requests for data are interspersed with work in the form of queries including joins, grouping and other database operations. Communications can be compared to the domain specific knowledge to determine whether the communications are part of a pattern of behavior that indicates an intrusion (i.e. an intrusion detection signature).
  • the communications may also be utilized to improve the model, dynamically update the signature, improve the domain specific knowledge and/or enhance other mechanism(s) employed for analysis.
  • This dynamic nature of the signatures, domain specific knowledge, model and/or detection techniques reduces the occurrence of false positives and generally improves detection of intruders.
  • communications from user(s) made in the course of authorized utilization of the data sources are intercepted at 1302 and analyzed at 1304.
  • the communications may be provided to the model as further examples of authorized use. Communications that are incorrectly determined to correspond to the intruder detection signature may be discovered upon further operation of the system. Consequently, the model is also updated to reduce the probability that a similar communication will result in an identification of the intruder detection signature.
  • a feedback loop may also be utilized to allow an administrator or other authorized entity for the system to confirm or identify communications that were incorrectly identified as intrusions.
  • communications that are attempts by attacker(s) to exfiltrate data and which correspond to the data discover signature are provided to the model to reinforce the detection of such intrusions.
  • the results of the statistical analysis described above may also be updated, for example to determine changes in patterns of communications. These changes in the patterns of behavior may themselves be correlated to intruder detection signatures and/or be utilized to update the intruder detection signature.
  • communications can be added to the domain specific knowledge that indicates authorized user behavior or the lack thereof.
  • communications are utilized not only for intrusion detection but also dynamically updating information regarding what type of communications constitutes an intrusion. Consequently, the mechanisms for determining using the communications to identify one or more intruder detection signatures is dynamically updated.
  • intrusion(s) are determined, at 1306. For example, if one or more the communications are considered to match or be part of a match for (e.g. identify) an intruder detection signature, an intrusion is detected at 1306.
  • a match need not be identical. Instead, a correlation as defined in the policies provided is sufficient.
  • the communication(s) need not be identical to a particular pattern within an intruder detection signature. Instead, the communication(s) may be considered to match the intruder detection signature if the communication(s) are sufficiently correlated with the intruder detection signature.
  • an intruder detection signature is a first set of communications corresponding to invalid login attempts for at least ten user accounts from a particular IP address on a first day followed by a second set of communications corresponding to invalid login attempts for at least nine of the same user accounts from a single IP address within thirty days
  • a set of communications that is at least ninety percent correlated is a match for the intruder detection signature.
  • an intruder detection signature may be considered to correspond to an n- dimensional vector in the space represented by the parameters for the intruder detection signature.
  • the communications having a corresponding vector within a particular distance of the n-dimensional vector for the intruder detection signature are considered to match the data discover signature.
  • the intruder detection signature vector has three dimensions corresponding to (1) a number of user accounts having invalid login attempts for the first day, (2) number of invalid login attempts from the same user accounts on the second day and (3) the number of days between the first day and the second day. If the communications correspond to a vector within a particular distance of the intruder detection signature vector in these three dimensions, then a match to the intruder detection signature is determined.
  • the particular distance may be set by policies and/or updated based upon the past behavior of the system.
  • other and/or additional mechanisms may be used to determine that communications are sufficiently correlated to intruder detection signature(s) to be a match.
  • the correlation between the intruder detection signature and the communications may also be determined statistically. In example above, if the communications are determined to cluster around intruder detection signature vector within the particular distance, then the communications are considered to be a match for the intruder detection signature. Similarly, in some embodiments, if the communications are in a Gaussian distribution centered around the intruder detection signature vector with a standard of deviation of not more than the particular distance, then the communications are considered to be a match for the intruder detection signature. Other statistical analyses and criteria may be used in some embodiments.
  • communications are compared to domain specific knowledge. If the comparison indicates that the communication is part of authorized behavior, then no intrusion is detected. If, however, the comparison indicates that the communication is part of unauthorized behavior, then the intrusion id detected. Thus an intrusion is determined at 1306.
  • the alarm is provided based on the intrusion being determined, at 1308.
  • the alarm may include a notification and/or action taken to prevent or reduce the exfiltration of data.
  • the alarm may include one or more of providing a warning to the network administrator or other entity tasked with managing security for the data sources, terminating the connection for the communication, temporarily blacklisting the client making the request, permanently blacklisting the client making the request, requiring the client to perform other tasks such as additional authentication procedures, suspending any queries already provided from the client making the request, and/or other responses to secure the integrity of the network.
  • the intrusion detected at 1306 may be addressed at 1308.
  • security may be improved.
  • sidecar 310 of system 300 depicted in FIG. 3 intercepts communications for data sources 302 and 304, at 1302.
  • dispatcher 312 intercepts all communications for data source(s) 302 and 304. Communications from clients 306-1, 306-2, 306-3 and 306-4 to data sources 302 and 304 are intercepted by dispatcher 312.
  • Dispatcher 312 provides copies of the intercepted communications to the appropriate service(s) 314 for analysis.
  • dispatcher 312 remains in step mode throughout its operation, holding communications for servicing by one or more services 314 before forwarding the communications to the appropriate data source(s) 302 and 304.
  • dispatcher 312 is only in step mode until the corresponding client 306 is authenticated. Dispatcher 312 then transitions to stream mode. In such an embodiment, communications are forwarded to the corresponding data source(s) 302 and 304 while services 314 perform their functions.
  • Services 314 analyze the communications to identify intruder detection signatures at 1304 and based on any intruder detection signatures identified, determine whether an intrusion has occurred.
  • Service(s) 314 may include or utilize models, domain specific knowledge and/or statistical methods in performing the analysis. The models may be part of the service(s) 314 or utilities 330-1 and/or 330-2.
  • Services 314 and/or dispatcher 312 provide the appropriate alarm in response to an intruder detection signature being detected. For example, service 314-1 or 314-2 may determine that an intruder detection signature exists for one or more communications from a client, such as client 306-4. Service 314-1 or 314-2 thus determines that an intrusion exists.
  • service 314-1 may consult policies and indicate that the intruder detection signature identified should result in the connection being terminated. This direction is provided from service 314-1 to dispatcher 312. In response, dispatcher 312 terminates the connection to the appropriate client 306.
  • the client carrying out the intrusion is blacklisted (prevented from accessing data sources 302 and/or 304) for a particular time interval. This time interval may be brief (e.g. minutes or hours), of intermediate length (e.g. hours or days) or permanent.
  • intrusions to a network including data sources such as data sources 302 and 304
  • data sources such as data sources 302 and 304
  • intruder detection signatures are utilized to discover the intrusions
  • such intrusions may be identified prior to a significant amount (or any) sensitive data being exfiltrated.
  • all communications for the data sources may be intercepted and monitored for intrusions, a large amount of information is available for analysis in method 1300.
  • method 1300 and system 300 may better protect data sources from threats.
  • OSI Layer 4 component e.g. a dispatcher
  • OSI layer 7 components e.g.
  • the analysis and identification may be temporally decoupled from flow of communications through the system.
  • protection against intruders may be achieved without significantly impacting performance. Further, this protection may be provided at or near real time. Thus, security may be improved substantially without adversely impacting performance of the system.
  • FIG. 14 is a flow chart depicting an embodiment of method 1400 for detecting intruders to a network.
  • Method 1400 is described in the context of system 300 (which, in some embodiments, may exclude collectors 320 for method 1400). Flowever, method 1400 may be used in connection with other systems including but not limited to systems 100 and 200. For simplicity, certain steps of method 1400 are depicted. Method 1400 may include other and/or additional steps and substeps. Further, the steps of method 1400 may be performed in another order, including performing portions or all of some steps in parallel. In some embodiments, method 1400 may be considered to be used in implementing 506 of method 500. Method 1400 may be considered to start after receipt of policies indicating how intrusions are to be treated.
  • Method 1400 is also utilized on communications that have been intercepted.
  • method 1400 may be utilized in performing 1304 of method 1300.
  • all communications are intercepted.
  • Method 1400 may thus be performed for all or only some of the communications for the network.
  • method 1400 may be used for multiple data sources.
  • a single service may perform method 1400.
  • multiple services may be utilized in performing method 1400.
  • service(s) may also employ utilities, such as utilities 330, in performing method 1400.
  • the behavioral baseline can include, be the same as, or be distinct from the behavioral baseline described in connection with methods 1200 and 1250 and collectors 320.
  • the behavioral baseline might include one or more of the context or pattern of communications from client(s), pattem(s) of data provided to or received from data sources, previous authentication attempts, other operations performed by the client(s) using the data sources, and/or other indicator of normal interaction with the data source(s).
  • the behavioral baseline is dynamically updated using the communications.
  • communications consistent with the behavioral baseline may be utilized to reinforce features of the behavioral baseline. Communications that do not match the behavioral baseline are used to provide indicators of interactions with the data source(s) that are outside the scope of normal, authorized behavior and thus help define the boundaries of the behavioral baseline. Communications that are incorrectly determined to match the behavioral baseline may be used to further refine the behavioral baseline. A determination that such a false positive has occurred may be made by an authorized user or another authorized entity. Consequently, in addition to being analyzed to determine whether they match the behavioral baseline, communications may also be used to dynamically update the behavioral baseline.
  • the behavioral baseline may be provided, utilized, and/or maintained by a model.
  • communications are identified as matching the intruder detection signature, at 1404.
  • communications that are inconsistent with the behavioral baseline are determined to correspond to the intruder detection signature described with respect to method 1300. Consequently, communications that do not match the behavioral baseline can be determined to correspond to intrusions.
  • Method 1300 may then be used to the appropriate alarm may then be provided.
  • communications for data sources 302 and 304 are intercepted by dispatcher 312. These communications may be copied and sent to service(s), such as service 314-1 and/or 314-2, for analysis. In some embodiments, the communications are also forwarded by dispatcher 312 for the appropriate data source(s) 304 and/or 304. In other embodiments, dispatcher 312 holds the communications until analysis is completed.
  • the communications are analyzed by services 314-1 and/or 314-2 to determine whether these communications match a behavioral baseline, at 1402.
  • Services 314- 1 and/or 314-2 may use a model, such as a hidden Markov model, to analyze the communications.
  • the model and other mechanisms utilized may be part of services 314-1 and/or 314-2 or may be part of one or more utilities 330.
  • the communications are from client(s) 320-1 through 320-3, the context provided by collectors 320 may be used as part of 1402. If the communications are from client 306-4, then the context may not be utilized.
  • communications from some or all of client(s) 306 may be analyzed to determine whether the data provided or received or other aspects of the communications match the behavioral baseline.
  • the communications are identified as corresponding to a particular intruder detection signature(s), at 1404.
  • intruder detection signatures can be identified and intrusions can be detected based on the behavioral baseline.
  • intrusions to a network including data sources may be discovered. Because intruder detection signatures are utilized to discover the intrusions, such intrusions may be identified before a significant amount (or any) sensitive data is exfiltrated. Because all communications for the data sources may be intercepted and monitored for intrusions, a large amount of information is available for analysis in method 1400. Thus, method 1400 and system 300 may better protect data sources from threats. Further, in embodiments in which intercepting communications is performed by an OSI Layer 4 component (e.g. a dispatcher) while analyzing the communications and identifying the intrusion are performed by OSI layer 7 components (e.g.
  • OSI Layer 4 component e.g. a dispatcher
  • the analysis and identification may be temporally decoupled from flow of communications through the system.
  • protection against intruders may be achieved without significantly impacting performance. Further, this protection may be provided at or near real time. Thus, security may be improved substantially without adversely impacting performance of the system.
  • port scanning may be detected using the techniques and mechanisms described herein.
  • an intruder may gain access to a host machine and maintain a presence in the network for a longer period of time. For example, an intruder may gain access to a user account by correctly guessing the user’s password. The intruder’s machine may then be considered an authorized client for the organization’s network. The intruder then endeavors, typically over the course of multiple days or weeks, to determine ports through which data sources can be accessed. To do so, an intruder provides communications that may attempt to determine the location of data sources and sensitive data. For example, the intruder may provide requests to send data to port 3306 (the default port for MySQL Protocol), port 5432 (a default TCP port) or other ports.
  • port 3306 the default port for MySQL Protocol
  • port 5432 a default TCP port
  • FIG. 15 is a flow chart depicting an embodiment of method 1500 for detecting intruders to a network.
  • Method 1500 may be used when port scanning is desired to be detected. For simplicity, certain steps of method 1500 are depicted. Method 1500 may include other and/or additional steps and substeps. Further, the steps of method 1500 may be performed in another order, including performing portions or all of some steps in parallel. In some embodiments, method 1500 may be considered to be used in implementing 506 of method 500.
  • Method 1500 may be considered to start after policies indicating how intrusions are to be treated are received.
  • method 1500 may be utilized in performing 1302 and 1304 of method 1300 and/or method 1400.
  • method 1500 may be used for multiple data sources.
  • a single service may perform method 1500.
  • multiple services may be utilized in performing method 1500.
  • service(s) may also employ utilities.
  • Communications including requests to send data to one or more ports are received, at 1502.
  • the ports are part of network(s) for an organization.
  • all communications for the data source(s) in the network(s) are intercepted.
  • 1502 is analogous to 1302.
  • the communications that are received and analyzed via method 1500 include the requests to upload data to particular ports.
  • the communications are analyzed to determine if signatures that correspond to an intruder detection signature are present in the communications and/or whether the communications match the behavioral baseline.
  • the behavioral baseline may be considered part of the domain specific knowledge regarding behavior of the system.
  • 1504 is analogous to 1304 and 1402. More specifically, both the data provided in the communications and aspects of the communication are analyzed at 1504. The analysis at 1504 may be carried out utilizing models and/or statistical analyses which are dynamically updated as described herein.
  • the data are analyzed to determine whether the data match a client signature for the port at 1504.
  • the client signature may be dynamically updated in a manner analogous to the behavioral baseline.
  • the client signature indicates data typically provided to the port and/or the data source accessed via the port. It may be determined at 1504 whether the type of data requested to be uploaded is sufficiently similar to data typically provided to the port. For example, if the port corresponds to a data source storing customer information, then the data provided in the requests may be analyzed to determine whether the format of the data matches that of customer names, customer addresses, credit card numbers and/or other customer information. If not, then this indicates that the user sending the data does not already know the type of data stored at the data source and may be an unauthorized user.
  • the invalid login attempts are required to be within a particular time interval in some embodiments.
  • invalid login attempts may be required to be clustered around or otherwise correlated with a particular time interval.
  • 1504 may include analyzing not only a request to provide data, but also other communications from the client requesting to provide data to the port. Multiple invalid login attempts may indicate that the corresponding password was guessed. Stated differently, the invalid login attempts may be analyzed to determine whether a credential guessing pattern is present. Thus, analyzing the communication from the client to determine whether the client had to repeatedly attempt to log in indicates that the client is a threat.
  • the client may be determined whether the client has made request(s) for data which the client is not authorized to access. Although the client has been authenticated for the network (possibly after repeated invalid login attempts), the client may be attempting to obtain data for which the corresponding account used is not authorized. In some embodiments, the client may not be authorized to access any portion of particular data set(s). In some embodiments, unauthorized access includes attempts to retrieve more data from a data set for which the client is authorized than is allowed. For example, a client may be allowed to access only some columns or only a particular number of rows at a time. The client may attempt to access other, restricted columns or more rows than is allowed.
  • communications are identified as having an intruder detection signature, at 1506.
  • only one of the failure to match the client signature, multiple invalid login attempts (matching a credential guessing pattern) and unauthorized requests for data (disallowed exfiltration pattern) is sufficient for the identification of the intruder detection signature.
  • some combination of data requested to be provided to port(s) not matching the client signature, multiple invalid login attempts from the client(s) requesting to provide data to port(s) and unauthorized requests to access data are used.
  • a threat such as port scanning is occurring if data that does not match the client signature is requested to be provided to particular ports in combination with multiple invalid login attempts are made for the client requesting uploading of the data.
  • the intruder detection signature such as a port scanning pattern
  • an intrusion can be identified. Consequently, an alarm can be provided using method 1300.
  • the connection to the client(s) making the requests may be terminated, the clients may be blacklisted, and/or other action may be taken.
  • FIG. 16 depicts system 1600 that may detect intruders to a network.
  • Method 1500 is described in the context of system 1600. Flowever, method 1500 may be used in connection with other systems including but not limited to systems 100, 200 and 300.
  • System 1600 is analogous to systems 100, 200 and 300 and includes components that are labeled similarly.
  • system 1600 includes data sources 1602-1, 1602-2 and 1604, clients 1606-1, 1606-2 and 1606-3 (collectively clients 1606) and sidecars 1610A and 1610B (collectively sidecars 1610). Although three data sources 1602-1, 1602-2 and 1604, three clients 1606 and two sidecars 1610 are shown, different numbers of data sources, clients, and/or sidecars may be present.
  • Data sources 1602-1, 1602-2 and 1604 and clients 1606 are analogous to data sources 102 and 104 and clients 106, respectively.
  • Sidecars 1610 are analogous to sidecars 110, 210 and 310.
  • sidecar 1610A includes dispatcher 1612A and services 1614-1A and 1614-2A (collectively services 1614A).
  • sidecar 1610B includes dispatcher 1612B and services 1614-1B, 1612-2B and 1614-3B (collectively services 1614B). Services 1614A and 1614B are collectively referred to as services 1614.
  • Sidecars 1610A and 1610B control accesses to data sources 1602-1, 1602-2 and 1604.
  • utilities may be present and may be used in performing method 1500 (e.g. may include models).
  • One or more of the services 1614 might be used in performing method 1600. Other services may also be provided. Two or more services may be used together in some embodiments. Also shown are listeners 1616-1A, 16161-2A, 1616-1B and 1616-2B (collectively listeners 1616). Listeners 1616 for ports (not explicitly shown) identify requests for data sources 1602-1, 1602-2 and 1604. [00143] Suppose that client 1606-2 is being used to attempt to exfiltrate sensitive data from data source(s) 1602-1, 1602-2 and 1604. Thus, a user of client 1606-2 may have correctly guessed the password for an existing account, including but not limited to the accounts corresponding to clients 1606-1 and/or 1606-3. Also assume services 1614-1 A and 1614-1B are used in performing method 1500.
  • Communications for data sources 1602-1, 1602-2 and 1604 are intercepted by dispatchers 1612, at 1502.
  • Listeners 1616 listen for requests from clients 1606 on ports.
  • Dispatchers 1612 intercept the received communications, at 1502.
  • dispatchers 1612 are in step mode throughout method 1500.
  • dispatchers 1612 utilize stream mode while used in method 1500.
  • Dispatchers 1612 copy the intercepted communications and provide the communications to services 1614.
  • dispatchers 1612 provide intercepted communications to services 1614- 1A and 1614-1B for analysis.
  • services 1614-1A and 1614-1B analyze the communications from clients 1606. Requests by clients 1606 to provide data to data sources 1602-1, 1602-2 and 1604 are analyzed to determine whether the data match a client signature for the corresponding port. Services 1614-1 A and 1614-1B may dynamically update the client signature. For requests from client 1606-2, the data may not match the client signature because client 1606-2 is utilized by an intruder who may not have knowledge of the types of data provided to data sources 1602-1, 1602-2 and 1604. However, for requests from clients 1606-1 and 1606-3, data may match the client signature because authorized users are utilizing clients 1606-1 and 1606-3.
  • services 1614-1A and 1614-1B determine whether communications from clients 1606 include multiple invalid login attempts (e.g. whether a credential guessing pattern is matched). Client 1606-2 is likely to have numerous invalid login attempts while the password for the account being hijacked is guessed. Thus, services 1614-1 A and 1614- IB determine that communications from client 1606-2 include multiple invalid login attempts. Services 1614-1A and 1614-1B also determine whether communications to or from clients 1606 involve attempts to accessed data for which clients 1606 are not authorized (e.g. whether a disallowed exfiltration pattern is matched). Suppose that client 1606-2 has not yet been able to access data sources 1602-1, 1602-2 and 1604. Consequently, client 1606-2 may not have attempted to exfiltrate data for which client 1606-2 is not authorized. Thus, services 1614-1A and 1614-1B may determine that client 1606-2 has not requested data for which client 1606-2 is not authorized.
  • Services 1614- 1A and 1614-1B ascertain whether the determinations made in 1504 correspond to intruder detection signatures, such as a port scanning pattern, at 1506.
  • Client 1606-2 has been determined by services 1614-1 A and 1614- IB to fail to match the client signature for data sources 1602-1, 1602-2 and 1604.
  • Client 1606-2 has been determined by services 1614-1A and 1614-1B to have multiple failed login attempts.
  • client 1606-2 may be determined by service(s) 1614-1A and/or 1614-1B to be engaged in a port scanning pattern.
  • service(s) 1614-1A and/or 1614-1B identify client 1606-2 as having an intruder detection signature, at 1506.
  • An intrusion by client 1606-2 is identified. It may be that client 1606-2 or 1606-3 has occasionally provided unusual data to data sources, had an invalid login attempt or attempted to access data for which the user is not authorized.
  • the intruder detection signatures, client signature, credential guessing pattern, disallowed exfiltration pattern and/or other patterns may be dynamically updated, as described above. Therefore, the probability that authorized clients 1606-2 and 1606-3 engage in such behavior with sufficient frequency for the intruder detection signature to be identified and for clients 1606-2 and 1606-3 to be determined to be intruders is low. Consequently, services 1614-1A and 1614-1B may correctly identify client 1606-2 as corresponding to an intruder and clients 1606-1 and 1606-3 as authorized clients.
  • intrusions such as port scanning may be discovered. Because client signatures and intruder detection signatures are utilized to discover the intrusions, such intrusions may be identified prior to a significant amount (or any) sensitive data being exfiltrated. Because all communications for data sources 1602-1, 1602-2 and 1604 may be monitored for intrusions, method 1500 and system 1600 may have access to more and/or improved information. Thus, method 1500 and system 1600 may better protect data sources 1602-1, 1602-2 and 1604 from threats. Further, dispatchers 1612 (e.g. OSI Layer 4 components) intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • OSI Layer 4 components intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • the analysis and identification may be temporally decoupled from flow of communications through the system.
  • protection against intruders may be achieved without significantly impacting performance. Further, this protection may be provided at or near real time. Security may, therefore, be improved substantially without adversely impacting performance of the system.
  • password spraying Another example of a potential threat to a security of a network and/or organization is password spraying.
  • Password spraying occurs when an intruder obtains access to account identifications (e.g. user identifications), but does not have the passwords for these accounts.
  • account identifications e.g. user identifications
  • an attacker may gain access to one or more host (e.g. clients) and attempt to access the network by logging into known accounts with a particular password or set of passwords. Thus, within a particular time interval, these known accounts have failed login attempts.
  • the attacker may be stymied by multi-factor authentication.
  • the MFA if present
  • password spraying is a type of intrusion that may be desirable to detect.
  • FIG. 17 is a flow chart depicting an embodiment of method 1700 for detecting intruders to a network.
  • Method 1700 may be used when password spraying is desired to be detected. For simplicity, certain steps of method 1700 are depicted. Method 1700 may include other and/or additional steps and substeps. Further, the steps of method 1700 may be performed in another order, including performing portions or all of some steps in parallel. In some embodiments, method 1700 may be considered to be used in implementing 506 of method 500. Method 1700 may be considered to start after policies indicating how intrusions are to be treated are received. Thus, in some embodiments, method 1700 may be utilized in performing 1302 and 1304 of method 1300 and/or method 1400.
  • method 1700 may be used for multiple data sources.
  • a single service may perform method 1700.
  • multiple services may be utilized in performing method 1700.
  • service(s) may also employ utilities.
  • Method 1700 is discussed in the context of system 1600. Flowever, method 1700 may be used with other systems, such as systems 100, 200 and/or 300.
  • 1702 includes intercepting the communications using a dispatcher.
  • the communications intercepted may include requests for authentication to the organization received from one or a few clients in some embodiments.
  • all communications for the data source(s) in the network(s) are intercepted.
  • 1702 is analogous to 1302.
  • the communications from one or more of the clients include repeated unsuccessful attempts to access the data source(s) using multiple data source access accounts (e.g. user accounts), at 1704.
  • it is determined whether a subset of clients generally not more than a few clients) repeatedly attempt to log in to multiple (sometimes numerous) accounts that can access the data source(s), but fail.
  • a subset of clients generally not more than a few clients
  • an attacker using particular client may attempt to utilize one or more passwords to access multiple known user accounts in a short window of time (e.g. on a particular day or within a few hours). If the password does not match any of the accounts, the attacker may later make another attempt with different password(s) for the same user accounts.
  • the attacker may still be defeated by MFA.
  • the attempts to access user accounts from one or a small number of clients may be unsuccessful either because the password failed or because MFA failed.
  • the attacker may retry with new passwords for the accounts for which the previous password failed and/or a different mechanism to defeat MFA.
  • the repeated unsuccessful attempts to access the data source(s) originate from one or a few clients and are for multiple (possibly a large number of) user accounts.
  • these repeated unsuccessful attempts to access the data source(s) are periodic in nature.
  • a cluster of multiple invalid attempts to log in to accounts may be centered around a particular time.
  • the cluster of invalid login attempts may be repeated, with the centers of the clusters separated by time intervals.
  • an attacker may make a few unsuccessful login attempts for each a large number of user accounts on a first day.
  • the attacker may make another set of unsuccessful login attempts for the user accounts three days layer.
  • the attacker may make a third set of unsuccessful login attempts for the user accounts two days later.
  • Such repeated invalid login attempts are considered periodic.
  • invalid login attempts may be analyzed (e.g. statistically analyzed) to determine whether such multiple invalid login attempts are periodic in nature and are associated with a few clients and multiple user accounts.
  • communications are identified as having an intruder detection signature, at 1706.
  • Communications determined at 1704 as being from a subset of the clients and involving repeated invalid attempts to access the data source are identified as matching intruder detection signatures, such as a password spraying pattern, at 1706.
  • the password spraying patern may be dynamically updated as part of 1704 and/or 1706. Consequently, an alarm can be provided using method 1300. For example, the connection to the client(s) making the authentication atempts may be terminated, the clients may be blacklisted, and/or other action may be taken.
  • client 1606-2 is being used to atempt to exfiltrate sensitive data from data source(s) 1602-1, 1602-2 and 1604.
  • client 1606-2 may have obtained user identification information for multiple existing accounts. Flowever, the passwords and other information allowing authentication have not been obtained.
  • services 1614- 1A and 1614-1B are used in performing method 1700.
  • dispatchers 1612 are in step mode throughout method 1700. In some embodiments, dispatchers 1612 utilize stream mode while used in method 1700. Dispatchers 1612 copy the intercepted communications and provide the communications to services 1614. At 1702, dispatchers 1612 provide intercepted communications to services 1614-1A and 1614-1B for analysis.
  • the communications intercepted at 1702 include communications to and from clients 1706 and include requests for authentication from client 1606-2 and responses from data sources 1602-1, 1602-2 and 1604 indicating login attempts were invalid.
  • services 1614-1A and 1614-1B analyze the communications. Requests by clients 1606 for authentication and the responses (e.g. indications of invalid login attempts) from data sources 1602-1, 1602-2 and/or 1604 or other components (not shown) of system 1600 are analyzed. Thus it is determined that client 1606-2 has made repeated invalid login attempts for multiple accounts.
  • Services 1614-lAand 1614-1B determine whether the determinations made in 1704 correspond to intruder detection signatures, such as a password spraying patern, at 1706.
  • Services 1614-1 A and 1614-1B may dynamically utilize models, domain specific knowledge, and/or statistical analysis to make the determinations in 1704 and 1706.
  • Client 1606-2 has been determined by services 1614-1 A and 1614- IB to match the password patern because of invalid login attempts (password mismatches and MFA failures). Thus, an intrusion by client 1606-2 is identified. Consequently, services 1614-1A and 1614- IB may correctly identify client 1606-2 as corresponding to an intruder.
  • intrusions such as password spraying for a network including data sources, such as data sources 1602-1, 1602-2 and 1604, may be discovered. Because password spraying patterns and intruder detection signatures are utilized to discover the intrusions, such intrusions may be identified prior to a significant amount (or any) sensitive data being exfiltrated. Because all communications for data sources 1602-1, 1602-2 and 1604 may be monitored for intrusions, more information is available for use. Thus, method 1700 and system 1600 may better protect data sources 1602-1, 1602-2 and 1604 from threats. Further, dispatchers 1612 (e.g. OSI Layer 4 components) intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • OSI Layer 4 components intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • the analysis and identification may be temporally decoupled from flow of communications through the system.
  • protection against intruders may be achieved without significantly impacting performance. Further, this protection may be provided at or near real time. Thus, security may be improved substantially without adversely impacting performance of the system.
  • trickle exfiltration Another example of a potential security threat is trickle exfiltration.
  • an intruder gains access to a data source in an apparently valid manner. The intruder then makes apparently legitimate requests for small amounts of sensitive data from the data source. For example, an attacker may obtain access to tables of sensitive data, such as credit card and/or social security numbers. The intruder then obtains from the table a few rows of data per request. Such requests may generally be considered valid as coming from an authenticated user. Over time, the intruder accumulates all of the sensitive data desired. Consequently, trickle exfiltration is a type of intrusion that may be desirable to detect.
  • FIG. 18 is a flow chart depicting an embodiment of method 1800 for detecting intruders to a network.
  • Method 1800 may be used when trickle exfiltration is desired to be detected. For simplicity, certain steps of method 1800 are depicted. Method 1800 may include other and/or additional steps and substeps. Further, the steps of method 1800 may be performed in another order, including performing portions or all of some steps in parallel. In some embodiments, method 1800 may be considered to be used in implementing 506 of method 500. Method 1800 may be considered to start after policies indicating how intrusions are to be treated are received. Method 1800 may be considered to start after communications (e.g. all communications) for data source(s) are intercepted.
  • communications e.g. all communications
  • method 1800 may be utilized in performing 1304 of method 1300 and/or method 1400. Although described in the context of access to a single data source, in some embodiments, method 1800 may be used for multiple data sources. In some embodiments, a single service may perform method 1800. In some embodiments, multiple services may be utilized in performing method 1800. Further, service(s) may also employ utilities. Method 1800 is discussed in the context of system 1600. However, method 1800 may be used with other systems, such as systems 100, 200 and/or 300.
  • Communications that have been intercepted are analyzed to determine whether the communications include multiple requests from particular client(s) for data from one or more data sets within particular time intervals, at 1802. For example, an attacker using particular client may request a few rows of data from a data set on a particular day. At another time (later in the day or another day), the particular client may request a small number of additional rows from the same data set. This type of request occurs repeatedly from the client. Requests from the client are analyzed at 1802 to identify this pattern of requests. The determination as to whether the requests match this pattern may be determined statistically. For example, it may be determined whether the requests cluster around one or more frequencies or whether the requests occur within a standard deviation of one or more dynamically updatable frequencies. In some embodiments, dynamically updated models are utilized for the determinations in 1802.
  • Communications are also analyzed to determine whether requests from the client(s) match a work signature, at 1804.
  • all requests from the client(s) are analyzed for the work signature.
  • only remaining requests not found at 1802 to be repeated requests are analyzed for the work signature.
  • the work signature is indicative of legitimate activities being carried out by the client and may be considered part of domain specific knowledge. For example, remaining requests from the client may be required to include actions other than requesting data.
  • the work signature may include requests for data from other data sets, join requests or other database operations, extract-transform-load (ETL) jobs, or other activities consistent with legitimate work being carried out by the client.
  • the work signature may be dynamically updated and may be used by a dynamically model, such as a hidden Markov model.
  • the determination as to whether the requests match the work signature may be determined statistically.
  • the work signature may be created for the data set using communications from a variety of clients.
  • communications are identified as having an intruder detection signature and/or not matching a behavioral baseline for the data source, at 1806.
  • the communications from the client(s) are identified as matching intruder detection signatures, such as a trickle exfiltration pattern, at 1806. Consequently, an alarm can be provided using method 1300. For example, the connection to the client(s) making the authentication attempts may be terminated, the clients may be blacklisted, and/or other action may be taken.
  • client 1606-2 is being used to attempt to obtain sensitive data from data source(s) 1602-1, 1602-2 and 1604.
  • an attacker using client 1606-2 may have obtained access to an existing account and has identified data sources 1602- 1, 1602-2 and 1604 as storing sensitive data that is desired to be requested.
  • services 1614-1A and 1614-1B are used in performing method 1800.
  • dispatchers 1612 are in step mode throughout method 1800. In some embodiments, dispatchers 1612 utilize stream mode while used in method 1800. Dispatchers 1612 copy the intercepted communications and provide the communications to services 1614. The communications intercepted include communications to and from clients 1706 and include apparently legitimate requests for data from data sources 1602-1, 1602-2 and 1604.
  • services 1614-1A and 1614-1B analyze the communications. Requests by clients 1606 for data and/or the responses (e.g. a few rows of data from a data set) from data sources 1602-1, 1602-2 and/or 1604 are analyzed. Thus it is determined at 1802 whether some of clients 1606 have made multiple requests for data from particular data sets within a particular time interval, such as a few days up to several weeks. Similarly, at 1804, requests from clients 1606 are analyzed to determine whether requests match a work signature. For example, if client 1606-2 only makes repeated requests to the data set(s) for the data, then the requests would not match a work signature.
  • Services 1614-lAand 1614-1B determine whether the determinations made in 1802 and 1804 correspond to intruder detection signatures, such as a trickle exfiltration pattern, at 1806.
  • Client 1606-2 has been determined by services 1614-1A and 1614-1B to match the trickle exfiltration pattern because of repeated requests for data which are not interspersed with requests that such that a work signature is matched.
  • an intrusion by client 1606-2 is identified. Consequently, services 1614-1A and 1614-1B may correctly identify client 1606-2 as corresponding to an intruder.
  • intrusions such as trickle exfiltration of data from a network including data sources, such as data sources 1602-1, 1602- 2 and 1604, may be discovered. Because trickle exfiltration patterns, work signatures and intruder detection signatures are utilized to discover the intrusions, such intrusions may be identified before a significant amount sensitive data is exfiltrated. Because all communications for data sources 1602-1, 1602-2 and 1604 may be monitored for intrusions, method 1800 and system 1600 may better protect data sources 1602-1, 1602-2 and 1604 from threats. Further, dispatchers 1612 (e.g. OSI Layer 4 components) intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • OSI Layer 4 components intercept communications while services 1614 (e.g. OSI Layer 7 components) analyze the communications and identify the intrusions.
  • the analysis and identification may be temporally decoupled from flow of communications through the system.
  • protection against intruders may be achieved without significantly impacting performance. Further, this protection may be provided at or near real time. Thus, security may be improved substantially without adversely impacting performance of the system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Storage Device Security (AREA)

Abstract

L'invention concerne une technique de détection d'intrus. Des communications pour une source de données dans une organisation sont interceptées et analysées pour identifier une signature de détection d'intrus. Une intrusion est déterminée sur la base de la signature de détection d'intrus et d'une alarme générée sur la base de l'intrusion.
PCT/US2020/043080 2019-08-22 2020-07-22 Détection d'intrus pour un réseau WO2021034441A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US16/548,732 2019-08-22
US16/548,732 US11477196B2 (en) 2018-09-18 2019-08-22 Architecture having a protective layer at the data source
US201962891795P 2019-08-26 2019-08-26
US62/891,795 2019-08-26
US16/900,619 US11477217B2 (en) 2018-09-18 2020-06-12 Intruder detection for a network
US16/900,619 2020-06-12

Publications (1)

Publication Number Publication Date
WO2021034441A1 true WO2021034441A1 (fr) 2021-02-25

Family

ID=74659723

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/043080 WO2021034441A1 (fr) 2019-08-22 2020-07-22 Détection d'intrus pour un réseau

Country Status (1)

Country Link
WO (1) WO2021034441A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020046275A1 (en) * 2000-06-12 2002-04-18 Mark Crosbie System and method for host and network based intrusion detection and response
US20100281541A1 (en) * 2004-05-11 2010-11-04 The Trustees Of Columbia University In The City Of New York Systems and Methods for Correlating and Distributing Intrusion Alert Information Among Collaborating Computer Systems
US7895649B1 (en) * 2003-04-04 2011-02-22 Raytheon Company Dynamic rule generation for an enterprise intrusion detection system
US20150033340A1 (en) * 2013-07-23 2015-01-29 Crypteia Networks S.A. Systems and methods for self-tuning network intrusion detection and prevention
US20150135293A1 (en) * 2011-09-08 2015-05-14 Drexel University Reconfigurable antenna based solutions for device authentication and intrusion detection in wireless networks
US20150350228A1 (en) * 2014-06-02 2015-12-03 Bastille Networks, Inc. Electromagnetic threat detection and mitigation in the internet of things
US9479532B1 (en) * 2013-07-16 2016-10-25 Go Daddy Operating Company, LLC Mitigating denial of service attacks

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020046275A1 (en) * 2000-06-12 2002-04-18 Mark Crosbie System and method for host and network based intrusion detection and response
US7895649B1 (en) * 2003-04-04 2011-02-22 Raytheon Company Dynamic rule generation for an enterprise intrusion detection system
US20100281541A1 (en) * 2004-05-11 2010-11-04 The Trustees Of Columbia University In The City Of New York Systems and Methods for Correlating and Distributing Intrusion Alert Information Among Collaborating Computer Systems
US20150135293A1 (en) * 2011-09-08 2015-05-14 Drexel University Reconfigurable antenna based solutions for device authentication and intrusion detection in wireless networks
US9479532B1 (en) * 2013-07-16 2016-10-25 Go Daddy Operating Company, LLC Mitigating denial of service attacks
US20150033340A1 (en) * 2013-07-23 2015-01-29 Crypteia Networks S.A. Systems and methods for self-tuning network intrusion detection and prevention
US20150350228A1 (en) * 2014-06-02 2015-12-03 Bastille Networks, Inc. Electromagnetic threat detection and mitigation in the internet of things

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ANWAR SHAHID, MOHAMAD ZAIN JASNI, ZOLKIPLI MOHAMAD FADLI, INAYAT ZAKIRA, KHAN SULEMAN, ANTHONY BOKOLO, CHANG VICTOR: "From Intrusion Detection to an Intrusion Response System: Fundamentals, Requirements, and Future Directions", ALGORITHMS, vol. 10, no. 2, 27 March 2017 (2017-03-27), XP055794265, Retrieved from the Internet <URL:https://www.mdpi.com/1999-4893/10/2/39> [retrieved on 20200911], DOI: 10.3390/a10020039 *

Similar Documents

Publication Publication Date Title
US11956235B2 (en) Behavioral baselining from a data source perspective for detection of compromised users
US11863557B2 (en) Sidecar architecture for stateless proxying to databases
US11836243B2 (en) Centralized applications credentials management
US20240031274A1 (en) Techniques for in-band topology connections in a proxy
US20240039914A1 (en) Non-in line data monitoring and security services
US11991192B2 (en) Intruder detection for a network
US20230334140A1 (en) Management of applications’ access to data resources
US20230198960A1 (en) Data masking
US20230065765A1 (en) Dynamic identity attribution
US20230061620A1 (en) Dynamic temporary data source access management
WO2021183278A1 (fr) Architecture de side-car pour la procuration sans état à des bases de données
US20240356937A1 (en) Intruder detection for a network
WO2021034441A1 (fr) Détection d&#39;intrus pour un réseau

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20855322

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20855322

Country of ref document: EP

Kind code of ref document: A1