US20220318332A1

US20220318332A1 - Intelligent naming of application program interfaces

Info

Publication number: US20220318332A1
Application number: US17/339,946
Authority: US
Inventors: Shubham Jindal; Avinash Kolluru; Ravindra Guntar; Anuj Toyal
Original assignee: Traceable Inc
Current assignee: Traceable Inc
Priority date: 2021-03-30
Filing date: 2021-06-05
Publication date: 2022-10-06
Also published as: US20220318081A1; US20220318618A1; US20220318378A1; US20220321587A1

Abstract

A system generates a number of uniform resource locators as application program interfaces in an intelligent way based on live traffic. Live traffic between a server and multiple users is monitored and intercepted and forwarded to a remote server that processes the traffic. Traffic URLs are processed to build a digital data structure that represents nodes or portions within each URL. For URLs having different nodes at the same hierarchical level that have the same type, the present system may replace those different nodes with a representative node. The representative node replaces the nodes at the same hierarchical level having the same type, but having different values.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the priority benefit of U.S. provisional patent application 63/167,649, filed on Mar. 30, 2021, titled “INTELLIGENT APPLICATION PROTECTION,” the disclosure of which IS incorporated herein by reference.

BACKGROUND

When analyzing, tracking, or monitoring a system, it is often important to know the specification of the system, including application program interface (API) specifications. However, when monitoring a new system, the API specification and related data is often unknown. This forces administrators and users working with the system to work with uniform resource locators (URL), and for example associate each URL with a particular API. This results in very large numbers of APIs, which requires a large number of resources to work with. What is needed is an improved method for naming APIs.

SUMMARY

The present technology, roughly described, renames a number of uniform resource locators as application program interfaces in an intelligent way based on live traffic. Live traffic between a server and multiple users is monitored and intercepted by an agent. The agent may be within the customer server, or positioned elsewhere to intercept traffic between users and the server. Intercepted traffic is forwarded to a remote server that processes the traffic.
Generating API names may be based on nodes in a URL. A URL may be considered to have a domain followed by a path. The path may have one or more portions, wherein each path portion begins with a forward slash. A digital data structure can be generated from a URL, wherein the domain and each path portion can correspond to a node in the digital data structure. For example, for the URL of /server/request/1234, the digital data structure can be considered to have a first node of server, a second note of request, and a third node of 1234. For URLs having matching nodes, the digital data structure represents the matching nodes as a single node. For URLs having nodes that are different, the different nodes may branch out from the previous node in the hierarchy. For example, for the URL of /server/request/123 and a URL of /server/request/234, the digital data structure may have base nodes of /server/request/and to third level notes that branch out from request and consist of 123 and 234.
Processing includes building a digital data structure, such as a trie, that represents nodes or portions generated from each URL. For URL based nodes at the same hierarchical level that have the same type, the present system may replace those different nodes with a representative node. The representative node replaces the nodes at the same hierarchical level having the same type, but having different values. In some instances, the replacement representative node is provided in response to detecting that specifications associated with the nodes being replaced are related or the same. For example, if the specifications for requests for the nodes have a similar format and specifications for the responses for nodes have a similar format, the nodes may be replaced with a replacement representative node.
In some instances, a method can generate application program interfaces from uniform resource locator information. The method can include receiving intercepted requests and responses sent to a URL address at the first server by a plurality of remote computing devices. The method continues with building a digital data structure based on the URLs to which the requests and the responses are sent, the digital data structure including a single node for identical portions of URLs located at the same hierarchical position within the digital data structure and including a separate node for different portions of URLs located at the same hierarchical position within the digital data structure, wherein each separate node is associated with a node type and a value. Specification data associated with a plurality of nodes which have the same hierarchical position, the same value type, and a different value can be compared. An API is generated based on two or more of the plurality of nodes which have the same hierarchical position and same value type, the new name based at least in part on the compared specification.
In some instances, a non-transitory computer readable storage medium has embodied thereon a program that is executable by a processor to perform a method. The method generates application program interfaces from uniform resource locator information. The method can include receiving intercepted requests and responses sent to a URL address at the first server by a plurality of remote computing devices. The method continues with building a digital data structure based on the URLs to which the requests and the responses are sent, the digital data structure including a single node for identical portions of URLs located at the same hierarchical position within the digital data structure and including a separate node for different portions of URLs located at the same hierarchical position within the digital data structure, wherein each separate node is associated with a node type and a value. Specification data associated with a plurality of nodes which have the same hierarchical position, the same value type, and a different value can be compared. An API is generated based on two or more of the plurality of nodes which have the same hierarchical position and same value type, the new name based at least in part on the compared specification.
In embodiments, a system can include a server, memory and one or more processors. One or more modules may be stored in memory and executed by the processors to receive intercepted requests and responses sent to a URL address at the first server by a plurality of remote computing devices, build a digital data structure based on the URLs to which the requests and the responses are sent, the digital data structure including a single node for identical portions of URLs located at the same hierarchical position within the digital data structure and including a separate node for different portions of URLs located at the same hierarchical position within the digital data structure, wherein each separate node is associated with a node type and a value, compare specification data associated with a plurality of nodes which have the same hierarchical position, the same value type, and a different value, and generate an API based on two or more of the plurality of nodes which have the same hierarchical position and same value type, the new name based at least in part on the compared specification.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 is a block diagram of a system for intelligently naming APIs.

FIG. 2 is a block diagram of an application that intelligently names APIs.

FIG. 3 is a method for intelligently naming APIs.

FIG. 4 is a method for building digital data structures from URLs.

FIG. 5 is a method for determining if digital data structures are associated with matching specifications.

FIG. 6 illustrates URLs from which an API can be intelligently named.

FIG. 7 illustrates a representation of a digital data structure.

FIG. 8 illustrates an intelligently named API based on the URLs of FIG. 6.

FIG. 9 is a block diagram of a system for implementing machines that implement the present technology.

DETAILED DESCRIPTION

The present system renames a number of uniform resource locators as application program interfaces in an intelligent way based on live traffic. Live traffic between a server and multiple users is monitored and intercepted by an agent. The agent may be within the customer server, or positioned elsewhere to intercept traffic between users and the server. Intercepted traffic is forwarded to a remote server that processes the traffic.
Generating API names may be based on nodes in a URL. A URL may be considered to have a domain followed by a path. The path may have one or more portions, wherein each path portion begins with a forward slash. A digital data structure can be generated from a URL, wherein the domain and each path portion can correspond to a node in the digital data structure. For example, for the URL of /server/request/1234, the digital data structure can be considered to have a first node of server, a second note of request, and a third node of 1234. For URLs having matching nodes, the digital data structure represents the matching nodes as a single node. For URLs having nodes that are different, the different nodes may branch out from the previous node in the hierarchy. For example, for the URL of /server/request/123 and a URL of /server/request/234, the digital data structure may have base nodes of /server/request/and to third level notes that branch out from request and consist of 123 and 234.
Processing includes building a digital data structure, such as a trie, that represents nodes or portions generated from each URL. For URL based nodes at the same hierarchical level that have the same type, the present system may replace those different nodes with a representative node. The representative node replaces the nodes at the same hierarchical level having the same type, but having different values. In some instances, the replacement representative node is provided in response to detecting that specifications associated with the nodes being replaced are related or the same. For example, if the specifications for requests for the nodes have a similar format and specifications for the responses for nodes have a similar format, the nodes may be replaced with a replacement representative node.
FIG. 1 is a block diagram of a system for intelligently naming (i.e., generating) APIs. The block diagram 100 of FIG. 1 includes client devices 110-140, customer server 150, network 160, API naming server 170, and data store 180.
Client devices 110-140 may send requests to and receive responses from customer server 150. The client devices may be any device which can access the service, network page, webpage, or other content from customer server 150. Client devices 110-140 may send a request to customer server 150, and customer server 150 may send a response to the devices based on the request. The request may be sent to a particular URL provided by customer server 150 in the sponsors may be sent from the same URL or different URLs. Though only for four client devices are shown, any number of client devices may be used to interact with customer server 150.
Customer server 150 may provide a service to client devices 110-140. Agent 152 on customer server 150 may monitor the communication between customer server 150 and client devices 110 - 140 and intercept traffic between the server and the devices. Upon intercepting the traffic, agent 152 may forward the traffic to application 172 on API naming server 170. In some instances, one or more agents may be installed on customer server 150, which may be implemented by one or more physical or logical machines. In some instances, server 150 may actually be implemented by multiple servers in different locations, providing a distributed service for devices 110-140. In any case, one or more agents 152 may be installed to intercept requests and responses sent between devices 110-140 and customer server 150 and for those requests and responses to application 172 on server 170.
Network 140 may include one or more private networks, public networks, intranets, the Internet, an intranet, wide-area networks, local area networks, cellular networks, radio-frequency networks, Wi-Fi networks, any other network which may be used to transmit data, and any combination of these networks. Client devices 110-140, customer server 150, API naming server 170, and data store 180 may all communicate over network 160.
API naming server 170 may be implemented as one or more physical or logical machines that provide API naming functionality as described herein. In some instances, API naming server may include one or more applications 172. The application 172 may be stored on one or more API naming servers 170 and be executed to perform functionality as described herein. API naming server and application 172 may both communicate over network 160 and data store 180. In some instances, data store 180 may include one or more URLs, generated API names, and other data.
FIG. 2 is a block diagram of an application that intelligently names APIs. Application 172 of FIG. 2 provides more detail for application 172 of the system of FIG. 1. Application 172 includes one or more modules, including digital data structure builder to 10, specification analyzer 220, node threshold engine 230, and API naming manager 240.
Digital data structure builder 210 may build a data structure from received URLs at application 172. Digital data structure may replace nodes, increment node counters, and otherwise build and manage digital data structures.
Specification analyzer 220 may analyze specifications related to intercepted URLs. In particular, specification analyzer 220 may access and compare the specification between URL requests, URL responses, and other data associated with a URL.
Node threshold engine 230 may determine if the number of occurrences of a particular node value and type at a particular position in a node hierarchy has exceeded a threshold associated with a particular node hierarchy position and type. The node threshold engine may maintain a counter, increment a counter, and flag nodes for which the threshold has or has not been exceeded.
API naming manager 240 may generate an API name based on URL similarity and specification analysis. API naming manager may generate the API name and store the API locally at server 170 or remotely at data store 180.
Application 172 may include more or fewer modules and the listed, and the modules as it may be grouped together or divided into multiple modules to implement the technology described herein.
FIG. 3 is a method for intelligently naming APIs. The method of FIG. 3 begins with installing an agent in a customer system to intercept customer traffic at step 310. Installing the agent may be performed remotely or locally by an admin. The agent may be installed, within the customer server or elsewhere, to intercept traffic between the server and computing devices associated with users of the server. Once installed, the agent may intercept traffic between the customer computing devices and the server.
Live traffic consisting of URL requests and responses sent between computing devices and a server are intercepted at step 320. In some instances, the computing devices may be associated with customers of the particular server. The request may be intercepted by one or more agents located within a customer server or otherwise positioned to intercept traffic between users of the server, for example from one or more computing devices to the customer servers. In some instances, intercepted traffic includes requests and responses sent to and from URLs supported by the customer server.
URL traffic is forwarded by the agent to the API naming server at step 330. The URL traffic may include a request, a corresponding response, the URL to which the request and response were sent to and from, and other data. In some instances, the agent may send the data in batches, periodically, or based on an event.
A digital data structure is built from URLs at step 340. A digital data structure may include nodes generated at least in part from or derived from received URLs. In some instances, the digital data structure may be a trie. In any case, the digital data structure may be built and updated as additional traffic information is received from one or more remote agents. More data for building digital data structures is discussed with respect to the method of FIG. 4.
A determination is made as to whether digital data structure nodes at the same level are associated with matching specifications at step 350. The specifications may be associated with nodes at the same level, and may be specifications related to a node request, response, or some other specification. Determining if digital data structure nodes at the same level in a URL hierarchy are associated with matching specifications is discussed in more detail with respect to the method of FIG. 5.
An API is renamed with related digital data structure nodes at the same level at step 360. Renaming an API with digital data structure nodes that are related may be performed if the digital data structure nodes represent the same type, are the same hierarchical level, and have specifications that match.
An example of APIs that are generated or renamed are illustrated in FIG. 6. The APIs 600 of FIG. 6 begin with a path of /api/profiles/. The next path portion in each URL of FIG. 6 is a number. The path portions after each number include text “view” and “requests.”
The digital data structure constructed from the URLs of FIG. 6 is displayed in FIG. 7. The numbers that make up the third node of the digital data structure can be examined to see if they're the same type (e.g., integer, GUID, etc.), and their related specifications match. With respect to the present example of FIGS. 6-7, it is presumed that the nodes have specifications that match. In this example, the API names generated from the URLs are those as illustrated in FIG. 8. FIGS. 6-8 are discussed in more detail below.
After renaming (e.g., generating) the APIs, the renamed APIs may be stored at step 370. The APIs may be stored locally at API naming server 170 or at a remote location, such as for example a data store 180.
FIG. 4 is a method for building digital data structures from URLs. The method of FIG. 4 provides more detail for step 340 of the method of FIG. 3. First, a digital data structure is built from the received URLs at step 410. The digital data structure may be built in any of several ways, including as a trie. A trie may have several nodes, each of which correspond to a portion of a URL path separated by a forward slash. An example of a trie built from a series of URLs is illustrated and discussed in more detail with respect to FIG. 7.
For nodes within a digital data structure that have the same hierarchical level and type but have different values, a node counter is incremented for the nodes at step 420. The node counter is used to track how many occurrences of a particular node occur in order to determine if that particular node should be combined to a single node within a named API.
A determination is made as whether the node counter exceeds a threshold for the node type at step 430. In some instances, the threshold may depend on the type associated with the node. For example, if a node has an integer type, the threshold may be 5, 8, 10, 12, or some other value between 5-25. If a node represents a GUID type, the threshold may also be somewhere between 5-25. If a node represents a string type, the threshold may be much larger, such as for example 100, 200 or even higher. In some instances, a threshold for any type may be as low as 2 or greater than 10,000, depending on design preferences. If the node counter is not exceeded a threshold for a node type at step 430, the method returns to step 420. If the node counter does exceed the threshold for the node type, then the particular note is flagged for a naming replacement at step 440.
FIG. 5 is a method for determining if digital data structures are associated with matching specifications. The method of FIG. 5 provides more detail for step 350 of the method of FIG. 3. URL requests are accessed for a digital data structure node level that satisfies a threshold at step 510. These URL requests include requests that were made to the URL by computing devices in communication with the server providing the URL. The access URL request specifications are then compared at step 250. A determination is then made if the request for stations match at step 530. The request specifications match if they include similar formats in the request header, request body, and other aspects of the request. If the request specifications do not match, then the request may be associated with different URLs, and should not be combined into a single API. In this case, the method of FIG. 5 continues to step 570 where the digital data structure nodes are determined to not match at step 570.
If the request specifications match, access URL response specifications are compared at step 540. The response specifications may be compared to determine if the response header, response body, and other portions of the responses match in content and/or format. If the response specifications match at step 550, the digital data structure nodes are determined to match at step 560. If the response specifications do not match at step 550, it is determined that the digital data structure nodes do not match at step 570.
FIG. 6 illustrates URLs from which an API can be intelligently named. The URLs of FIG. 6 can be provided by a server which receives requests and provides responses to computing devices accessing services provided by the server. The URLs each include a path that contains “/api/profiles/” but have subsequent portions of a path that differ. A digital data structure for the URLs in FIG. 6 is shown in FIG. 7.
FIG. 7 illustrates a representation of a digital data structure. Although the digital data structure of FIG. 7 is a trie, other types of digital data structures can be used to organize URL paths into nodes. In the trie 700 of FIG. 7, the top or parent node is API, and the second note is profiles, corresponding to the first portions of the path of the URLs in FIG. 6. From the profiles note, there are three child nodes, 123, 234, and 456. These correspond to the different sub paths from the “profiles” path portion. The child node 123 includes child nodes of view and requests, the child node 234 includes a child node of view, and the child node of 456 includes a child node of requests.
Though the values of the child nodes below the child node profile have different numbers, they are all of the same type: integers. As such, they can be combined as a replacement type in a named API. From that replacement type node, the four remaining child nodes have two different values. As such, APIs may be named from the trie illustrated in FIG. 7, each of which include replacement type after the node “profiles.” The two named APIs are illustrated in FIG. 8.
FIG. 9 is a block diagram of a system for implementing machines that implement the present technology. System 900 of FIG. 9 may be implemented in the contexts of the likes of machines that implement client devices 110-140, customer server 150, API naming server 170, and data store 180. The computing system 900 of FIG. 9 includes one or more processors 910 and memory 920. Main memory 920 stores, in part, instructions and data for execution by processor 910. Main memory 920 can store the executable code when in operation. The system 900 of FIG. 9 further includes a mass storage device 930, portable storage medium drive(s) 940, output devices 950, user input devices 960, a graphics display 970, and peripheral devices 980.
The components shown in FIG. 9 are depicted as being connected via a single bus 990. However, the components may be connected through one or more data transport means. For example, processor unit 910 and main memory 920 may be connected via a local microprocessor bus, and the mass storage device 930, peripheral device(s) 980, portable storage device 940, and display system 970 may be connected via one or more input/output (I/O) buses.
Mass storage device 930, which may be implemented with a magnetic disk drive, an optical disk drive, a flash drive, or other device, is a non-volatile storage device for storing data and instructions for use by processor unit 910. Mass storage device 930 can store the system software for implementing embodiments of the present invention for purposes of loading that software into main memory 920.
Portable storage device 940 operates in conjunction with a portable non-volatile storage medium, such as a floppy disk, compact disk or Digital video disc, USB drive, memory card or stick, or other portable or removable memory, to input and output data and code to and from the computer system 900 of FIG. 9. The system software for implementing embodiments of the present invention may be stored on such a portable medium and input to the computer system 900 via the portable storage device 940.
Input devices 960 provide a portion of a user interface. Input devices 960 may include an alpha-numeric keypad, such as a keyboard, for inputting alpha-numeric and other information, a pointing device such as a mouse, a trackball, stylus, cursor direction keys, microphone, touch-screen, accelerometer, and other input devices. Additionally, the system 900 as shown in FIG. 9 includes output devices 950. Examples of suitable output devices include speakers, printers, network interfaces, and monitors.
Display system 970 may include a liquid crystal display (LCD) or other suitable display device. Display system 970 receives textual and graphical information and processes the information for output to the display device. Display system 970 may also receive input as a touch-screen.
Peripherals 980 may include any type of computer support device to add additional functionality to the computer system. For example, peripheral device(s) 980 may include a modem or a router, printer, and other device.
The system of 900 may also include, in some implementations, antennas, radio transmitters and radio receivers 990. The antennas and radios may be implemented in devices such as smart phones, tablets, and other devices that may communicate wirelessly. The one or more antennas may operate at one or more radio frequencies suitable to send and receive data over cellular networks, Wi-Fi networks, commercial device networks such as a Bluetooth device, and other radio frequency networks. The devices may include one or more radio transmitters and receivers for processing signals sent and received using the antennas.
The components contained in the computer system 900 of FIG. 9 are those typically found in computer systems that may be suitable for use with embodiments of the present invention and are intended to represent a broad category of such computer components that are well known in the art. Thus, the computer system 900 of FIG. 9 can be a personal computer, handheld computing device, smart phone, mobile computing device, workstation, server, minicomputer, mainframe computer, or any other computing device. The computer can also include different bus configurations, networked platforms, multi-processor platforms, etc. Various operating systems can be used including Unix, Linux, Windows, Macintosh OS, Android, as well as languages including Java, .NET, C, C++, Node.JS, and other suitable languages.
The foregoing detailed description of the technology herein has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen to best explain the principles of the technology and its practical application to thereby enable others skilled in the art to best utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the technology be defined by the claims appended hereto.

Claims

What is claimed is:

1. A method for generating application program interfaces (APIs) from uniform resource locator (URL) information, comprising:

receiving intercepted requests and responses sent to a URL address at the first server by a plurality of remote computing devices;

building a digital data structure based on the URLs to which the requests and the responses are sent, the digital data structure including a single node for identical portions of URLs located at the same hierarchical position within the digital data structure and including a separate node for different portions of URLs located at the same hierarchical position within the digital data structure, wherein each separate node is associated with a node type and a value;

comparing specification data associated with a plurality of nodes which have the same hierarchical position, the same value type, and a different value; and

generating an API based on two or more of the plurality of nodes which have the same hierarchical position and same value type, the new name based at least in part on the compared specification.

2. The method of claim 1, wherein the specification has the same format for each of the two or more of the plurality of nodes which have the same hierarchical position and same value type.

3. The method of claim 1, wherein the specification data for the plurality of nodes which have the same hierarchical position and the same value type includes a specification for a request sent to a URL associated with the node.

4. The method of claim 1, wherein the specification data for the plurality of nodes which have the same hierarchical position and the same value type includes a specification for a response sent from a URL associated with the node.

5. The method of claim 1, wherein the digital data structure is a trie

6. The method of claim 1, further comprising:

determining if the number of nodes which have the same hierarchical position and the same value type exceed a threshold number of nodes; and

comparing the specification data if the number of nodes exceeds the threshold.

7. The method of claim 6, wherein the threshold is greater than five nodes having a numerical value type.

8. The method of claim 6, wherein the threshold is greater than 5 for nodes having a unique identifier type.

9. The method of claim 1, wherein the requests and responses are intercepted by an agent installed on a first server, the building, comparing and generating performed by an application on a second server in response to the agent transmitting the URL requests and responses to the application.

10. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for generating application program interfaces (APIs) from uniform resource locator (URL) information, the method comprising:

11. The non-transitory computer readable storage medium of claim 10, wherein the specification has the same format for each of the two or more of the plurality of nodes which have the same hierarchical position and same value type.

12. The non-transitory computer readable storage medium of claim 10, wherein the specification data for the plurality of nodes which have the same hierarchical position and the same value type includes a specification for a request sent to a URL associated with the node.

13. The non-transitory computer readable storage medium of claim 10, wherein the specification data for the plurality of nodes which have the same hierarchical position and the same value type includes a specification for a response sent from a URL associated with the node.

14. The non-transitory computer readable storage medium of claim 10, wherein the digital data structure is a trie.

15. The non-transitory computer readable storage medium of claim 10, the method further comprising:

comparing the specification data if the number of nodes exceeds the threshold.

16. The non-transitory computer readable storage medium of claim 15, wherein the threshold is greater than 5 for nodes having a numerical value type.

17. The non-transitory computer readable storage medium of claim 15, wherein the threshold is greater than 5 for nodes having a unique identifier type.

18. The non-transitory computer readable storage medium of claim 10, wherein the requests and responses are intercepted by an agent installed on a first server, the building, comparing and generating performed by an application on a second server in response to the agent transmitting the URL requests and responses to the application.

19. A system for generating application program interfaces (APIs) from uniform resource locator (URL) information, comprising:

a server including a memory and a processor; and

one or more modules stored in the memory and executed by the processor to receive intercepted requests and responses sent to a URL address at the first server by a plurality of remote computing devices, build a digital data structure based on the URLs to which the requests and the responses are sent, the digital data structure including a single node for identical portions of URLs located at the same hierarchical position within the digital data structure and including a separate node for different portions of URLs located at the same hierarchical position within the digital data structure, wherein each separate node is associated with a node type and a value, compare specification data associated with a plurality of nodes which have the same hierarchical position, the same value type, and a different value, and generate an API based on two or more of the plurality of nodes which have the same hierarchical position and same value type, the new name based at least in part on the compared specification.

20. The system of claim 19, wherein the specification has the same format for each of the two or more of the plurality of nodes which have the same hierarchical position and same value type.