US20200341956A1 - Processing time series metrics data - Google Patents
Processing time series metrics data Download PDFInfo
- Publication number
- US20200341956A1 US20200341956A1 US16/396,516 US201916396516A US2020341956A1 US 20200341956 A1 US20200341956 A1 US 20200341956A1 US 201916396516 A US201916396516 A US 201916396516A US 2020341956 A1 US2020341956 A1 US 2020341956A1
- Authority
- US
- United States
- Prior art keywords
- resolution
- metric
- tsm
- resolution database
- data points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
Definitions
- One or more implementations relate to high volume data processing in a computing system, and more specifically, to processing time series metrics data.
- a time series is a series of numeric data points of a metric over time. Each time series consists of the metric plus metadata associated with this metric.
- a metric is any particular measurement of data to be tracked over time. For example, a metric in the context of a cloud computing server may be determining the number of operations accessing a system resource by a set of users per unit time.
- Time series metrics data points are often stored in a computer database.
- the database may be resident on a storage device such as a hard disk drive (HDD) or a solid-state drive (SSD).
- HDD hard disk drive
- SSD solid-state drive
- a typical write strategy for storing the time series metrics data points results in highly random disk input/output (I/O) operations.
- I/O disk input/output
- FIG. 1 is a diagram of a system for processing time series metrics data according to an embodiment.
- FIG. 2 is a flow diagram of storing time series metrics data points according to an embodiment.
- FIG. 3 is a flow diagram of staging time series metrics data points according to an embodiment.
- FIG. 4 is a diagram of sharding time series metrics data according to an embodiment.
- FIG. 5 is a diagram of restart processing according to some embodiments.
- FIG. 6 is a flow diagram of restart processing according to some embodiments.
- FIG. 7 illustrates an electronic device according to some implementations.
- Embodiments of the present invention provide high performance read and write throughput and support writing millions of TSM data points and serve millions of TSM data points per minute with low query latency.
- Embodiments of the present invention “sequentialize” the random write input/output (I/O) operations typically generated by TSM data processing systems.
- TSM data processing systems write one or more files for each metric to one or more storage devices (e.g., HDDs and/or SSDs). These files are updated according to the frequency of the arriving metrics data. In some scenarios, the amount of data processed is on the order of millions of TSM data points per minute.
- a TSM data processing system typically needs to write millions of files to storage in a random order. On aggregation these files need to be rewritten as well when a higher resolution of certain TSM data gets transformed into a lower resolution, causing even more data storage churn.
- Embodiments of the present invention use a database embedded in the TSM data processor system process to store all TSM data points once per interval and in different buckets for different resolutions, thereby avoiding churn during aggregation.
- a Log-Structured-Merge (LSM) tree data structure is used to store TSM data points.
- TSM data points arrive at regular intervals, such as every minute. Since the arrival order of TSM data points is non-deterministic, using an LSM tree to store these TSM data points helps to ensure that TSM data points are written out in batches into the C1 structure of the LSM tree.
- TSM data points instead of organizing TSM data points by time, TSM data points are co-located by series when stored. This allows the LSM tree to store the TSM data points more effectively.
- the LSM tree database is a key/value store.
- a key is defined as (“metric-id”, “timestamp”) and a TSM data point value is the value of the record.
- one TSM data point per metric is received every minute.
- the LSM tree accumulates data in memory and then writes the data to disk files “as-is”, resulting fast storage rates.
- Embodiments of the present invention write data into the LSM tree as the TSM data points arrive instead of finding the right place for each TSM data point in the LSM tree at the time each data point is written (using slower random disk I/O).
- the files are processed by the LSM tree to organize records by key at a later time in batches (resulting in faster sequential disk I/O).
- TSM data processing systems typically aggregate sets of higher resolution TSM data points into sets of lower resolution TSM data points at predefined intervals. This aggregation operation results in reading and writing files entirely every time an interval boundary is reached.
- embodiments of the present invention use separate groups of TSM data points for different aggregation resolutions. TSM data points are moved into lower resolution groups that are stored separately from higher resolution groups.
- embodiments build indexes on top of the LSM tree to speed up query processing as well as to perform validation.
- Embodiments support user queries by fully-qualified metric namespaces, including the use of wildcards.
- embodiments implement indexes built on top of the TSM data points and the metric names. These indexes are logical indexes, that is they are built on top of the LSM tree database for this specific purpose.
- Embodiments provide an integrated solution that operates standalone without the requirement of an external database. This minimizes operational efforts and allows for straightforward scaling of the TSM data points processing.
- Embodiments of the present invention also provide support for “sharding” by metric namespace to facilitate scaling out of disk writes.
- Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards.
- Metrics namespaces are used as sharding keys.
- Embodiments maintain full compatibility with existing open source TSM systems without any changes being needed to interface with existing implementations.
- Embodiments provide support for restarts and outages of the storage backend without losing data points through the use of a queue instead of a classical forwarding of data.
- embodiments prioritize processing of live real-time data while catching up in the background on TSM data points that were not yet processed due to a restart or outage and thus not persisted to disk.
- the latest TSM data points will be available right away for further processing while the gap is being backfilled.
- embodiments can recover even from sustained outage periods while still providing real-time data immediately. This is advantageous for computing systems providing real-time alerts derived from processing these TSM data points. Otherwise, the real-time alerts would be rendered useless until the TSM data processing is completely caught up.
- Embodiments of the present invention thus provide an advantage over how other time series databases typically handle these recovery scenarios.
- FIG. 1 is a diagram of a system 100 for processing time series metrics (TSM) data according to an embodiment.
- TSM data processor 104 comprises metrics controller 105 and storage engine 106 .
- a TSM data point comprises a metric name (e.g., text such as ⁇ a.b.c.d ⁇ ), a timestamp (e.g., Unix epoch time), and a value (e.g., a floating-point value such as 1.02).
- TSM data processor 104 is executed in a process within a JavaTM virtual machine (JVM) hosted on a computer system (e.g., a computer server in a cloud computing data center).
- JVM JavaTM virtual machine
- TSM data processor 104 includes storage engine 106 to temporarily store TSM data points (e.g., within main memory (random-access memory (RAM) of a host computer system).
- storage engine 106 includes an LSM tree data structure to persistently store key-value pairs.
- storage engine 106 comprises an open source embedded RocksDB storage engine implemented in the C++ programming language wherein keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function.
- storage engine 106 is embedded in the JavaTM process running TSM data processor 104 using a JavaTM Native Interface (JNI) library to take advantage of a shared memory space. In other embodiments, other databases may be used.
- JNI JavaTM Native Interface
- TSM data processor 104 also stores information in file storage 108 .
- file storage 108 is comprised of one or more of HDDs, SSDs, or a combination of HDDs and SSDs, coupled with TSM data processor 104 using known means (e.g., a communications bus, etc.).
- TSM data processor 104 manages read and write data access to TSM data points stored in file storage 108 .
- the accesses are optimized for sequential I/O operations.
- File storage 108 includes resolution databases 110 , which includes a plurality of databases for storing sets of TSM data points according to a selected resolution.
- Each entry in a resolution database includes a key field and a value field.
- Each resolution database is indexed by a key comprising two integers—a metric identifier (ID) and a timestamp.
- a resolution database entry value field stores a TSM data point value.
- resolution databases 100 includes first database (DB) 1 112 at a first resolution, second database DB 2 114 at a second resolution lower than the first resolution, . . . DB N 116 at an Nth resolution, the Nth resolution lower than the first resolution, the second resolution, and any intervening resolutions, where N is a natural number.
- TSM data processor 104 never reads resolution databases 110 as part of the TSM data points storage processing.
- Embodiments avoid any reads or updates on the existing time series data. This is different from typical time series databases that would try to put multiple values for the same metric within the same record (e.g., a lookup and update will be needed). These prior solutions try to optimize for space by using different approaches to compact a sequence of time/values.
- embodiments of the present invention avoid random access I/O operations (e.g., record lookup or update) so resolution database records are “immutable” and accessed only when time series data is queried. Resolution databases are not queried during a write path.
- Each persistence policy defines rules for storing TSM data points.
- a persistence policy includes one or more attributes such as a rule for matching a metric name (e.g., *.ecom.*, for example, using “* as a wildcard matching operation), and a rule for specifying time resolution and retention (e.g., 60 seconds/24 hours, 5 minutes/7 days, or 30 minutes/2 years).
- FIG. 2 is a flow diagram 200 of storing time series metrics data points according to an embodiment.
- metrics controller 105 receives a TSM data point. TSM data points are sent by servers and server processes in regular intervals stating the current value of a given characteristic of a system or subsystem. For example, a TSM data point could be the current number of active users on an ecommerce system. This data can then be visualized to determine the state of health of such a system or subsystem.
- metrics controller 105 gets a persistence policy 124 from file storage 108 using the metric name obtained from the received TSM data point. The persistent policy 124 specifies the time resolution and retention to be used for the received TSM data point.
- metrics controller 105 stores the received TSM data point in the selected staging file 122 for the second resolution database identified by the staging file name (e.g., stores the metric ID of the received TSM data point and the TSM data point's value as text in a line of the selected staging file).
- a TSM data point is written in two places: the first (e.g., current) resolution database and the staging file accumulating TSM data points for the second (e.g., next) resolution database.
- This processing repeats for all supported resolutions. Processing stops when the staging file for the last defined resolution is processed.
- a TSM data point is written directly to a resolution database only for the highest resolution database. In all other cases (e.g., for lower resolutions) the TSM data point that is stored in a resolution database calculated from the staging file.
- a new staging file is created by metrics controller 105 for the same time interval and both files are sorted and merged together into a single staging file.
- a selected staging file 122 is sorted based at least in part on a metric ID to group all TSM data points for the same metric.
- the staging file includes TSM data points for all metrics for that time interval. For example, for a resolution database with a 5 min resolution, there are 12 separate staging files over 1 hour, and for a resolution database with a 30 minute resolution, there are 2 separate staging files over 1 hour.
- each line in staging file 122 includes a TSM data point. If there is data still to be processed in the staging file 122 , then at block 308 metrics controller 105 loads a successive group of TSM data points from the staging file (e.g., successive lines of data in the file) until a next metric ID is encountered in the file. At block 310 , metrics controller 105 gets the metric name using the metric ID (e.g., based on metric ID to name index 120 and metric name index 118 ). In an embodiment, using the metric ID instead of the metric name saves disk space.
- metrics controller 105 applies an aggregation rule defined for the metric name.
- Aggregation rules can be configured and are stored on disk. Aggregation can be done differently for each metric on a per-name basis. For example, metric “a.b.c” can have its TSM data points averaged on aggregation while metric “a.b.d” could have its TSM data points summarized. The individual TSM data points for a given time period (e.g., the 5 data points for metric “a.b.c.” that arrived over the last 5 minutes) will be merged into one data point “a.b.c” with the value being the average or sum of the 5 individual data points.
- metrics controller 105 stores the aggregated TSM data points for the group in the appropriate resolution database that applies for this staging file (such as a second resolution database).
- metrics controller 105 selects a third (e.g., new next) resolution database 110 (e.g., a lower resolution database than the current one, that is, one of DB 1 112 , DB 2 114 , . . . DB N 116 ) for the group of TSM data points using the persistence policy 124 associated with the metric.
- metrics controller 105 at block 320 stores the aggregated TSM data points of the group in the staging file 122 associated with the third (e.g., new next) resolution database. Processing continues with the next group (if any) of the original staging file at block 304 . If the third resolution database is not defined at block 318 , processing continues with the next group (if any) of the original staging file at block 304 .
- FIG. 4 is a diagram 400 of sharding according to an embodiment.
- Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards.
- the word shard means a small part of a whole.
- each shard is held on a separate database server instance, to spread load.
- a shard contains a complete and independent instance of the TSM data processor and associated storage in file storage 108 (e.g., includes an instance of metric name index 118 , metric ID to name index 120 , staging files 122 , persistence policies 124 , and resolution databases 110 ).
- Each shard has its own, independent resolution databases and sharding files appropriate to the subset of metric names associated with the shard.
- two instances of TSM data processor 104 are contained in a shard to provide for redundancy in case of system outages.
- TSM data points 402 , 404 , 406 , . . . 408 are received by metrics controller 105 .
- Each TSM data point is associated with a metric name.
- TSM data point 402 is named XX1.metricname
- TSM data point 404 is named XX2.metricname
- TSM data point 406 is named XX3.metricname
- TSM data point 408 is named XXZ.metricname, where Z is a natural number.
- metrics controller 105 stores TSM data points in shards in file storage 108 .
- TSM data point XX1.metricname 402 is stored in shard 1 412 , since shard 1 412 is associated with a first metric name, TSM data point XX2.metricname 402 is stored in shard 2 414 , since shard 2 414 is associated with a second metric name, TSM data point XX3.metricname 406 is stored in shard 3 414 , since shard 3 414 is associated with a third metric name, . . . TSM data point XXZ.metricname 408 is stored in shard Z 418 , since shard Z 418 is associated with a “Z'th” metric name.
- Sharding is used in embodiments of the present invention to scale metrics controller processing and file storage over many computer servers in many data centers.
- shard 1 412 may be used to store TSM data points for data center pods 1 through 10
- shard 2 414 may be used to store TSM data points for data center pods 11 through 20 , and so on.
- User input 420 is received by web renderer 422 to retrieve information based at least in part on TSM data points stored in file storage 108 .
- Web renderer 422 has to query all known shards for the given TSM data points and time frames.
- Web renderer 422 has no knowledge of where the requested data is stored nor does it have a need for this information.
- Web renderer 422 can effectively query all shards in parallel and merge the resulting data set from all shards into a consistent reply to the user input query.
- Web renderer 422 forwards the information to dashboard 424 , which formats the information for display as user output 426 .
- FIG. 5 is a diagram of restart processing according to some embodiments.
- metrics controller 105 includes recovery metrics processor 520 to process TSM data points from metrics queue 502 during a restart, and real time metrics processor 522 to process TSM data points from metrics queue 502 while the system is operational. Both metrics processors 520 , 522 perform the operations disclosed above with reference to FIGS. 1-4 .
- Metrics controller 105 also includes common metrics processing framework 524 , which provides common functions used by recovery metrics processor 520 and real time metrics processor 522 to store TSM data points in file storage 108 as described above.
- real time metrics processor 522 is executed a higher priority process in a computing device and recovery metrics processor 520 is executed at a lower priority process in the computing device.
- TSM data points are forwarded from computing devices that generate the TSM data points into metrics queue 502 .
- metrics queue 502 is stored in a memory.
- TSM data points are stored in the metrics queue in time order, left to right, according to FIG. 5 .
- Metrics data received before a system outage occurs include metrics queue entries 504 and 506 .
- TSM data points 504 are stored in metrics queue 502 before a last checkpoint is performed at 514 .
- TSM data points 506 include data stored in the metrics queue from the start of the outage to the start of recovery processing. Recovery processing starts with the next TSM data points received after the last checkpoint at 514 .
- Metrics data received during the system outage include metrics queue entries 508 and 510 .
- Recovery metrics processor 520 of metrics controller 105 processes TSM data points from metrics queue entries 506 and 508 .
- Arrow 516 points to a current position in the metrics queue that recovery metrics processor 520 is processing at a point in time.
- metrics data received after the outage includes block 512 .
- real time metrics processor 522 of metrics controller 105 processes TSM data points stored in metric queue entries 512 .
- Arrow 518 points to a current position in the metrics queue that real time metrics processor 522 is processing at the point in time.
- Metrics queue entries 510 are TSM data points that were received during the system outage but were not processed by real time metrics processor 522 , and they have not yet been processed by recovery metrics processor 520 . Note that TSM data points 506 get processed twice: once before restart by real time metrics processor 522 and by recovery metrics processor 520 after the restart.
- TSM data processor can provide up to date metrics for users which include TSM data points received during and after the outage.
- recovery metrics processor 520 reads the last checkpoint position from file storage 108 .
- recovery metrics processor creates a gap.
- the last check point position is the start of the gap and the current time is the end time of the gap.
- the checkpoint position of the gap is initialized to the start time of the gap. This is added to the list of gaps to be processed.
- the gap is defined by the start timestamp, end timestamp, and checkpoint timestamp.
- a checkpoint is created within a gap so as to recover from situations where the system was restarted while it was recovering from an outage.
- recovery processing ends at block 630 .
- TSM data point is read from file storage 108 .
- recovery metrics processor 520 reads a TSM data point from the gap's last checkpoint in metrics queue 502 .
- TSM data point's time is not less than the gap's end time, then processing of the gap is complete and the processed gap is deleted at block 628 . Processing continues in that case with block 614 .
- recovery metrics processor 520 processes the TSM data point at block 622 .
- the gap's checkpoint is updated.
- recovery metrics processor gets the next TSM data point from metrics queue 502 for the gap. Processing continues with block 620 .
- an electronic device may include a non-volatile memory (e.g., phase change memory) to store the code/data when the electronic device is turned off, and that same non-volatile memory has sufficiently fast read/write times such that, rather than copying the part of the code to be executed into volatile memory, the code/data may be provided directly to the processor(s) (e.g., loaded into a cache of the processor(s)); in other words, this non-volatile memory operates as both long term storage and main memory, and thus the electronic device may have no or only a small amount of DRAM for main memory.
- Typical electronic devices also include a set of one or more physical network interface(s) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices.
- FIG. 7 illustrates an electronic device 704 according to some implementations.
- FIG. 7 includes hardware 740 comprising a set of one or more processor(s) 742 , a set or one or more network interfaces 744 (wireless and/or wired), and non-transitory machine-readable storage media 748 having stored therein software 750 .
- TSM data processor 104 may be implemented in one or more electronic devices 704 .
- each one of a plurality of consumer devices is implemented in a separate one of the electronic devices 704 (e.g., in an end user electronic device operated by an end user; in which case, the software 750 in each such end user electronic device includes the software to implement one of the end user clients, including software to interface with a cloud computing service (e.g., an application programming interface (API), a web browser, a native client, a portal, a command-line interface, etc.)) and TSM data processor 104 is implemented in a separate set of one or more of the electronic devices 704 (in which case, the software 750 is the software to implement TSM data point processing as described herein; in operation, the end user electronic devices and the electronic device(s) implementing the cloud computing system containing TSM data processor 104 would be commutatively coupled (e.g., by a network) and would establish between them connections for requesting and providing cloud computing services.
- Other configurations of electronic devices may be used in other implementations.
- the processor(s) 742 typically execute software to instantiate a virtualization layer 754 and software container(s) 762 A-R (e.g., with operating system-level virtualization, the virtualization layer 754 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple software containers 762 A-R (representing separate user space instances and also called virtualization engines, virtual private servers, or jails) that may each be used to execute a set of one or more applications; with full virtualization, the virtualization layer 754 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system (OS), and the software containers 762 A-R each represent a tightly isolated form of software container called a virtual machine that is run by the hypervisor and may include a guest operating system; with para-virtualization, an operating system or application running with a virtual machine may be aware of the presence of virtualization for
- VMM virtual machine monitor
- an instance of the software 750 (illustrated as instance 776 A) is executed within the software container 762 A on the virtualization layer 754 .
- instance 776 A on top of a host operating system is executed on the “bare metal” electronic device 704 .
- the instantiation of the instance 776 A, as well as the virtualization layer 754 and software containers 762 A-R if implemented, are collectively referred to as software instance(s) 552 .
- references in the specification to “one implementation,” “an implementation,” “an example implementation,” “some implementations,” etc., indicate that the implementation described may include a particular feature, structure, or characteristic, but every implementation may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an implementation, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other implementations whether or not explicitly described.
- Bracketed text and blocks with dashed borders may be used herein to illustrate optional operations and/or structures that add additional features to some implementations. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain implementations.
- Coupled is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other.
Abstract
Description
- One or more implementations relate to high volume data processing in a computing system, and more specifically, to processing time series metrics data.
- A time series is a series of numeric data points of a metric over time. Each time series consists of the metric plus metadata associated with this metric. A metric is any particular measurement of data to be tracked over time. For example, a metric in the context of a cloud computing server may be determining the number of operations accessing a system resource by a set of users per unit time.
- Time series metrics data points are often stored in a computer database. The database may be resident on a storage device such as a hard disk drive (HDD) or a solid-state drive (SSD). A typical write strategy for storing the time series metrics data points results in highly random disk input/output (I/O) operations. When the number of time series metrics data points per unit time grows very large, as in the case for some cloud computing data center metrics, the random nature of these I/O operations causes the storage of the time series metrics data points to scale poorly and become unsustainable.
- The following figures use like reference numbers to refer to like elements. Although the following figures depict various exemplary implementations, alternative implementations are within the spirit and scope of the appended claims. In the drawings:
-
FIG. 1 is a diagram of a system for processing time series metrics data according to an embodiment. -
FIG. 2 is a flow diagram of storing time series metrics data points according to an embodiment. -
FIG. 3 is a flow diagram of staging time series metrics data points according to an embodiment. -
FIG. 4 is a diagram of sharding time series metrics data according to an embodiment. -
FIG. 5 is a diagram of restart processing according to some embodiments. -
FIG. 6 is a flow diagram of restart processing according to some embodiments. -
FIG. 7 illustrates an electronic device according to some implementations. - The following description describes a method and apparatus for implementing a time series metrics (TSM) data store using Log-Structured-Merge (LSM) trees and co-locating data by series to perform sequential disk writes. This results in a scalable and efficient way of storing the TSM data on inexpensive magnetic disks and supporting loss-less restarts without external dependencies. Embodiments of the present invention provide high performance read and write throughput and support writing millions of TSM data points and serve millions of TSM data points per minute with low query latency.
- Embodiments of the present invention “sequentialize” the random write input/output (I/O) operations typically generated by TSM data processing systems. TSM data processing systems write one or more files for each metric to one or more storage devices (e.g., HDDs and/or SSDs). These files are updated according to the frequency of the arriving metrics data. In some scenarios, the amount of data processed is on the order of millions of TSM data points per minute. A TSM data processing system typically needs to write millions of files to storage in a random order. On aggregation these files need to be rewritten as well when a higher resolution of certain TSM data gets transformed into a lower resolution, causing even more data storage churn. Embodiments of the present invention use a database embedded in the TSM data processor system process to store all TSM data points once per interval and in different buckets for different resolutions, thereby avoiding churn during aggregation.
- In an embodiment, a Log-Structured-Merge (LSM) tree data structure is used to store TSM data points. In an embodiment, TSM data points arrive at regular intervals, such as every minute. Since the arrival order of TSM data points is non-deterministic, using an LSM tree to store these TSM data points helps to ensure that TSM data points are written out in batches into the C1 structure of the LSM tree. In an embodiment, instead of organizing TSM data points by time, TSM data points are co-located by series when stored. This allows the LSM tree to store the TSM data points more effectively.
- In embodiments, the LSM tree database is a key/value store. A key is defined as (“metric-id”, “timestamp”) and a TSM data point value is the value of the record. In an embodiment, one TSM data point per metric is received every minute. The LSM tree accumulates data in memory and then writes the data to disk files “as-is”, resulting fast storage rates. Embodiments of the present invention write data into the LSM tree as the TSM data points arrive instead of finding the right place for each TSM data point in the LSM tree at the time each data point is written (using slower random disk I/O). The files are processed by the LSM tree to organize records by key at a later time in batches (resulting in faster sequential disk I/O). One trade-off is that a read operation is more expensive since the LSM tree needs to check multiple files for a requested TSM data point. But in case of a time series database (such as the LSM tree used herein), there are a lot more writes than reads. The record key that is chosen to use ensures that in the LSM tree files TSM data points that belong to the same metric will be stored in the same file next to each other so the LSM tree will be able to process read requests such as “find data points for metric A for the last 24 hours” will be processed more quickly than in other approaches. Also, the LSM tree of embodiments of the present invention is designed to be embedded into various types of applications and exposes a variety of different tuning parameters.
- TSM data processing systems typically aggregate sets of higher resolution TSM data points into sets of lower resolution TSM data points at predefined intervals. This aggregation operation results in reading and writing files entirely every time an interval boundary is reached. In contrast, embodiments of the present invention use separate groups of TSM data points for different aggregation resolutions. TSM data points are moved into lower resolution groups that are stored separately from higher resolution groups.
- In order to be able to efficiently retrieve TSM data points from the LSM tree, embodiments build indexes on top of the LSM tree to speed up query processing as well as to perform validation. Embodiments support user queries by fully-qualified metric namespaces, including the use of wildcards. To effectively query using these inputs, embodiments implement indexes built on top of the TSM data points and the metric names. These indexes are logical indexes, that is they are built on top of the LSM tree database for this specific purpose.
- Although it may be possible in some embodiments to use an alternate storage backend solution that also uses an LSM tree for storing TSM data, this results in the presence of an external database running on another computing cluster in a large cloud computing data center. This requires additional computing resources in the data center and results in data storage delays to write the TSM data into other cluster members. Embodiments provide an integrated solution that operates standalone without the requirement of an external database. This minimizes operational efforts and allows for straightforward scaling of the TSM data points processing.
- Embodiments of the present invention also provide support for “sharding” by metric namespace to facilitate scaling out of disk writes. Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards. Metrics namespaces are used as sharding keys. Embodiments maintain full compatibility with existing open source TSM systems without any changes being needed to interface with existing implementations.
- Embodiments provide support for restarts and outages of the storage backend without losing data points through the use of a queue instead of a classical forwarding of data. After startup, embodiments prioritize processing of live real-time data while catching up in the background on TSM data points that were not yet processed due to a restart or outage and thus not persisted to disk. The latest TSM data points will be available right away for further processing while the gap is being backfilled. In this way embodiments can recover even from sustained outage periods while still providing real-time data immediately. This is advantageous for computing systems providing real-time alerts derived from processing these TSM data points. Otherwise, the real-time alerts would be rendered useless until the TSM data processing is completely caught up. Embodiments of the present invention thus provide an advantage over how other time series databases typically handle these recovery scenarios.
-
FIG. 1 is a diagram of asystem 100 for processing time series metrics (TSM) data according to an embodiment. A plurality of TSM data points 102 are received and processed byTSM data processor 104.TSM data processor 104 comprisesmetrics controller 105 andstorage engine 106. A TSM data point comprises a metric name (e.g., text such as {a.b.c.d}), a timestamp (e.g., Unix epoch time), and a value (e.g., a floating-point value such as 1.02). In an embodiment,TSM data processor 104 is executed in a process within a Java™ virtual machine (JVM) hosted on a computer system (e.g., a computer server in a cloud computing data center). In an embodiment, there may be multiple instances ofTSM data processor 104 running on one or more host computer systems.Metrics controller 105 controls processing of TSM data points as discussed below with reference toFIGS. 2 through 6 .TSM data processor 104 includesstorage engine 106 to temporarily store TSM data points (e.g., within main memory (random-access memory (RAM) of a host computer system). In an embodiment,storage engine 106 includes an LSM tree data structure to persistently store key-value pairs. In one embodiment,storage engine 106 comprises an open source embedded RocksDB storage engine implemented in the C++ programming language wherein keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function. In an embodiment,storage engine 106 is embedded in the Java™ process runningTSM data processor 104 using a Java™ Native Interface (JNI) library to take advantage of a shared memory space. In other embodiments, other databases may be used. -
TSM data processor 104 also stores information infile storage 108. In an embodiment,file storage 108 is comprised of one or more of HDDs, SSDs, or a combination of HDDs and SSDs, coupled withTSM data processor 104 using known means (e.g., a communications bus, etc.).TSM data processor 104 manages read and write data access to TSM data points stored infile storage 108. In an embodiment, the accesses are optimized for sequential I/O operations. -
File storage 108 includesresolution databases 110, which includes a plurality of databases for storing sets of TSM data points according to a selected resolution. Each entry in a resolution database includes a key field and a value field. Each resolution database is indexed by a key comprising two integers—a metric identifier (ID) and a timestamp. A resolution database entry value field stores a TSM data point value. For example,resolution databases 100 includes first database (DB) 1 112 at a first resolution,second database DB 2 114 at a second resolution lower than the first resolution, . . .DB N 116 at an Nth resolution, the Nth resolution lower than the first resolution, the second resolution, and any intervening resolutions, where N is a natural number. Each resolution database is specified by a resolution and retention pair of values. The resolution defines how often TSM data points are received and the retention defines how long the TSM data points are accumulated. For example, firstresolution database DB 1 112 could have a resolution of 60 seconds and a retention of 24 hours, meaning that TSM data points 102 are received for a selected metric once per minute and are accumulated inDB 1 112 for 24 hours (e.g., resulting in 60*60*24=86,400 stored TSM data points). For example, secondresolution database DB 2 114 could have a resolution of five minutes and a retention of seven days, meaning that TSM data points 102 are received for a selected metric once every five minutes and are accumulated inDB 2 114 for seven days (e.g., resulting in 12*24*7=2,016 stored TSM data points). For example, an Nth resolutiondatabase DB N 116 could have a resolution of 30 minutes and a retention of two years, meaning that TSM data points 102 are received once per half hour and are accumulated inDB N 116 for two years (e.g., resulting in 2*24*365*2=35,400 stored TSM data points). In various embodiments there can be any number of resolution databases storing any combination of resolution and retention pairs. - In an embodiment,
TSM data processor 104 never readsresolution databases 110 as part of the TSM data points storage processing. Embodiments avoid any reads or updates on the existing time series data. This is different from typical time series databases that would try to put multiple values for the same metric within the same record (e.g., a lookup and update will be needed). These prior solutions try to optimize for space by using different approaches to compact a sequence of time/values. In contrast, embodiments of the present invention avoid random access I/O operations (e.g., record lookup or update) so resolution database records are “immutable” and accessed only when time series data is queried. Resolution databases are not queried during a write path. - In an embodiment,
resolution databases 110 are stored in a RocksDB database. In other embodiments, other databases may be used.File storage 108 includes metric name index 118 (e.g., name-index/). Embodiments map hierarchical metric names into a tree structure similar to a file system structure with non-leaf nodes (e.g., directories) and leaf nodes (e.g., files). For example, two metric names “a.b.c” and “a.b.d” will be stored in four entries in metric name index 118: “a”->“b”->“c”, “d”. In an embodiment, metric name is a text string.File storage 108 includes metric ID to name index 120 (e.g., id-index/). Embodiments use metric ID to nameindex 120 to map a metric ID to a metric name. In an embodiment, metric ID is an integer. In an embodiment,metric name index 118 and metric ID to nameindex 120 are stored in a RocksDB database. In other embodiments, other databases may be used.File storage 108 includes staging files 122. In an embodiment, staging files are text files used to accumulate TSM data points to be aggregated. The use of text files allowsmetrics controller 105 to employ existing highly efficient Linux™ file sort operations (in one embodiment) on staging files 122. In an embodiment, each line of a staging file includes a TSM data point having a metric ID and a value.File storage 108 also includes one ormore persistence policies 124. Each persistence policy defines rules for storing TSM data points. In an embodiment, a persistence policy includes one or more attributes such as a rule for matching a metric name (e.g., *.ecom.*, for example, using “* as a wildcard matching operation), and a rule for specifying time resolution and retention (e.g., 60 seconds/24 hours, 5 minutes/7 days, or 30 minutes/2 years). -
FIG. 2 is a flow diagram 200 of storing time series metrics data points according to an embodiment. Atblock 202,metrics controller 105 receives a TSM data point. TSM data points are sent by servers and server processes in regular intervals stating the current value of a given characteristic of a system or subsystem. For example, a TSM data point could be the current number of active users on an ecommerce system. This data can then be visualized to determine the state of health of such a system or subsystem. Atblock 204,metrics controller 105 gets apersistence policy 124 fromfile storage 108 using the metric name obtained from the received TSM data point. Thepersistent policy 124 specifies the time resolution and retention to be used for the received TSM data point. Atblock 206,metrics controller 105 gets a metric ID using the metric name. In an embodiment,metrics controller 105 accesses metric ID tometric name index 120 infile storage 108 to obtain the metric ID associated with the metric name for the received TSM data point. In an embodiment, the metric ID is local to an instance ofTSM data processor 104. Atblock 208,metrics controller 105 checks if the metric ID exists. If the metric ID does not yet exist, atblock 210 metrics controller creates a new entry in metric ID to nameindex 120, populates the metric name field of the entry with the metric name of the received TSM data point, and uses the metric ID of the new entry for further processing of the received TSM data point. Atblock 212,metrics controller 105 uses thepersistence policy 124 for the metric obtained atblock 204 to select a first resolution database (e.g., one ofDB 1 112,DB 2 114, . . . DB N 116) fromresolution databases 110 in which to store the received TSM data point. Atblock 214,metrics controller 105 stores the received TSM data point in the first resolution database. Atbock 216,metrics controller 105 uses thepersistence policy 124 for the metric to select a second (e.g., next, higher resolution) resolution database (e.g., one ofDB 1 112,DB 2 114, . . .DB N 116, but not one chosen at block 212). If the selected second database is not defined atblock 218, processing of the received TSM data point ends atblock 224. If the selected second database is defined atblock 218, thenmetrics controller 105 determines a selectedstaging file 122 name for also storing the received TSM data point. The selected staging file is used to store TSM data points to be stored in the second (e.g., next higher resolution) resolution database. In an embodiment, the staging file name encodes a target resolution database name and retention time interval. For example, for a received TSM data point with a timestamp=1553897370 and a resolution time interval=1553897370−(1553897370% (5*60)), the file name could be “5m7d-1553897100”. Atblock 222,metrics controller 105 stores the received TSM data point in the selectedstaging file 122 for the second resolution database identified by the staging file name (e.g., stores the metric ID of the received TSM data point and the TSM data point's value as text in a line of the selected staging file). - Thus, at every iteration a TSM data point is written in two places: the first (e.g., current) resolution database and the staging file accumulating TSM data points for the second (e.g., next) resolution database. This processing repeats for all supported resolutions. Processing stops when the staging file for the last defined resolution is processed. In an embodiment, a TSM data point is written directly to a resolution database only for the highest resolution database. In all other cases (e.g., for lower resolutions) the TSM data point that is stored in a resolution database calculated from the staging file. For example, if the first (e.g., highest) resolution is 1 minute and the next resolution is 5 minutes, then when the TSM data point arrives the TSM data point will be stored in the first resolution database with a 1 minute resolution and stored in a staging file accumulating data for the second resolution database with a 5 minute resolution. Then 5 to 6 minutes later the staging file will be processed. TSM data points that have to be stored in the 5 minute database are calculated and an aggregation function is applied (based on five 1 minute TSM data points that are in the staging file). The calculated TSM data points are stored in the 5 minute resolution database, and then these TSM data points will also be stored in a staging file for the next defined resolution (for example, a 30 minute resolution database).
- Processing of
blocks 202 to 224 are repeated for every TSM data point received by metrics controller 10. As can be appreciated, the size ofresolution databases 110 and stagingfiles 122 may grow very large over time, resulting in performance loads on devices (e.g., HDDs and/or SSDs) implementingfile storage 108. In an embodiment, this fact is an accepted compromise, and the use of disk space is sacrificed in return for improved write performance. -
FIG. 3 is a flow diagram 300 of staging time series metrics data points according to an embodiment. Staging is performed at the end of a resolution time interval (e.g., every 1 minute for the 60 seconds/24 hour resolution, and every 5 minutes for the 5 minutes/7 days resolution and so on). In an embodiment, astaging file 122 name comprises aresolution database 110 name and an interval timestamp to use for all TSM data points stored therein.Staging file 122 content comprises TSM data points that belong to the time interval defined by the timestamp and the resolution. In an embodiment, TSM data points are added to astaging file 122 bymetrics controller 105 only until the staging file is sorted. After that, if delayed TSM data points arrive, a new staging file is created bymetrics controller 105 for the same time interval and both files are sorted and merged together into a single staging file. Atblock 302, a selectedstaging file 122 is sorted based at least in part on a metric ID to group all TSM data points for the same metric. In an embodiment, there is one staging file per time interval of a resolution. The staging file includes TSM data points for all metrics for that time interval. For example, for a resolution database with a 5 min resolution, there are 12 separate staging files over 1 hour, and for a resolution database with a 30 minute resolution, there are 2 separate staging files over 1 hour. After sorting, if there is no data yet to be processed in the staging file, processing ends ablock 306. In an embodiment, each line in stagingfile 122 includes a TSM data point. If there is data still to be processed in thestaging file 122, then atblock 308metrics controller 105 loads a successive group of TSM data points from the staging file (e.g., successive lines of data in the file) until a next metric ID is encountered in the file. Atblock 310,metrics controller 105 gets the metric name using the metric ID (e.g., based on metric ID to nameindex 120 and metric name index 118). In an embodiment, using the metric ID instead of the metric name saves disk space. - At
block 312,metrics controller 105 applies an aggregation rule defined for the metric name. Aggregation rules can be configured and are stored on disk. Aggregation can be done differently for each metric on a per-name basis. For example, metric “a.b.c” can have its TSM data points averaged on aggregation while metric “a.b.d” could have its TSM data points summarized. The individual TSM data points for a given time period (e.g., the 5 data points for metric “a.b.c.” that arrived over the last 5 minutes) will be merged into one data point “a.b.c” with the value being the average or sum of the 5 individual data points. - At
block 314,metrics controller 105 stores the aggregated TSM data points for the group in the appropriate resolution database that applies for this staging file (such as a second resolution database). Atblock 316,metrics controller 105 selects a third (e.g., new next) resolution database 110 (e.g., a lower resolution database than the current one, that is, one ofDB 1 112,DB 2 114, . . . DB N 116) for the group of TSM data points using thepersistence policy 124 associated with the metric. If the third (e.g., new next) resolution database is defined atblock 318, thenmetrics controller 105 atblock 320 stores the aggregated TSM data points of the group in thestaging file 122 associated with the third (e.g., new next) resolution database. Processing continues with the next group (if any) of the original staging file atblock 304. If the third resolution database is not defined atblock 318, processing continues with the next group (if any) of the original staging file atblock 304. -
FIG. 4 is a diagram 400 of sharding according to an embodiment. Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards. The word shard means a small part of a whole. In some embodiments, each shard is held on a separate database server instance, to spread load. In an embodiment, a shard contains a complete and independent instance of the TSM data processor and associated storage in file storage 108 (e.g., includes an instance ofmetric name index 118, metric ID to nameindex 120, stagingfiles 122,persistence policies 124, and resolution databases 110). Each shard has its own, independent resolution databases and sharding files appropriate to the subset of metric names associated with the shard. In an embodiment, two instances ofTSM data processor 104 are contained in a shard to provide for redundancy in case of system outages. - TSM data points 402, 404, 406, . . . 408 are received by
metrics controller 105. Each TSM data point is associated with a metric name. For example,TSM data point 402 is named XX1.metricname,TSM data point 404 is named XX2.metricname,TSM data point 406 is named XX3.metricname, . . .TSM data point 408 is named XXZ.metricname, where Z is a natural number. As part ofmetrics controller 105 processing, in anembodiment metrics controller 105 stores TSM data points in shards infile storage 108. For example, TSM datapoint XX1.metricname 402 is stored inshard 1 412, sinceshard 1 412 is associated with a first metric name, TSM datapoint XX2.metricname 402 is stored inshard 2 414, sinceshard 2 414 is associated with a second metric name, TSM datapoint XX3.metricname 406 is stored inshard 3 414, sinceshard 3 414 is associated with a third metric name, . . . TSM datapoint XXZ.metricname 408 is stored inshard Z 418, sinceshard Z 418 is associated with a “Z'th” metric name. - Sharding is used in embodiments of the present invention to scale metrics controller processing and file storage over many computer servers in many data centers. For example,
shard 1 412 may be used to store TSM data points fordata center pods 1 through 10,shard 2 414 may be used to store TSM data points for data center pods 11 through 20, and so on. -
User input 420 is received byweb renderer 422 to retrieve information based at least in part on TSM data points stored infile storage 108.Web renderer 422 has to query all known shards for the given TSM data points and time frames.Web renderer 422 has no knowledge of where the requested data is stored nor does it have a need for this information.Web renderer 422 can effectively query all shards in parallel and merge the resulting data set from all shards into a consistent reply to the user input query.Web renderer 422 forwards the information todashboard 424, which formats the information for display asuser output 426. -
FIG. 5 is a diagram of restart processing according to some embodiments. In an embodiment,metrics controller 105 includesrecovery metrics processor 520 to process TSM data points frommetrics queue 502 during a restart, and realtime metrics processor 522 to process TSM data points frommetrics queue 502 while the system is operational. Bothmetrics processors FIGS. 1-4 .Metrics controller 105 also includes commonmetrics processing framework 524, which provides common functions used byrecovery metrics processor 520 and realtime metrics processor 522 to store TSM data points infile storage 108 as described above. In an embodiment, realtime metrics processor 522 is executed a higher priority process in a computing device andrecovery metrics processor 520 is executed at a lower priority process in the computing device. - TSM data points are forwarded from computing devices that generate the TSM data points into
metrics queue 502. In an embodiment,metrics queue 502 is stored in a memory. TSM data points are stored in the metrics queue in time order, left to right, according toFIG. 5 . Metrics data received before a system outage occurs include metrics queueentries metrics queue 502 before a last checkpoint is performed at 514. TSM data points 506 include data stored in the metrics queue from the start of the outage to the start of recovery processing. Recovery processing starts with the next TSM data points received after the last checkpoint at 514. Metrics data received during the system outage include metrics queueentries Recovery metrics processor 520 ofmetrics controller 105 processes TSM data points frommetrics queue entries Arrow 516 points to a current position in the metrics queue thatrecovery metrics processor 520 is processing at a point in time. When the system outage is over, metrics data received after the outage includesblock 512. Upon restart, realtime metrics processor 522 ofmetrics controller 105 processes TSM data points stored inmetric queue entries 512.Arrow 518 points to a current position in the metrics queue that realtime metrics processor 522 is processing at the point in time.Metrics queue entries 510 are TSM data points that were received during the system outage but were not processed by realtime metrics processor 522, and they have not yet been processed byrecovery metrics processor 520. Note that TSM data points 506 get processed twice: once before restart by realtime metrics processor 522 and byrecovery metrics processor 520 after the restart. - Because upon restart
recovery metrics processor 520 processes TSM data points received (but not processed) during the outage in parallel with realtime metrics processor 522 processing newly arriving TSM data points after the restart, TSM data processor can provide up to date metrics for users which include TSM data points received during and after the outage. -
FIG. 6 is a flow diagram 600 of restart processing according to some embodiments. Atblock 602, restart processing begins. Atblock 604, realtime metrics processor 522 reads a TSM data point from the restart position (e.g., the starting point of section 512) inmetrics queue 502. Atblock 606, real time metrics processor processes the TSM data point. Atblock 608, real time metrics processor checkpoints the processed TSM data point intofile storage 108. Realtime metrics processor 522repeats blocks - At block 610 (in parallel with performance of
blocks 604 through 608),recovery metrics processor 520 reads the last checkpoint position fromfile storage 108. Atblock 612, recovery metrics processor creates a gap. The last check point position is the start of the gap and the current time is the end time of the gap. The checkpoint position of the gap is initialized to the start time of the gap. This is added to the list of gaps to be processed. The gap is defined by the start timestamp, end timestamp, and checkpoint timestamp. A checkpoint is created within a gap so as to recover from situations where the system was restarted while it was recovering from an outage. Atblock 614, if no TSM data points from the gap are left to process, then recovery processing ends atblock 630. If there are TSM data points from the gap left to process, processing continues withblock 616, where gap information is read fromfile storage 108. Atblock 618,recovery metrics processor 520 reads a TSM data point from the gap's last checkpoint inmetrics queue 502. Atblock 620, if the TSM data point's time is not less than the gap's end time, then processing of the gap is complete and the processed gap is deleted atblock 628. Processing continues in that case withblock 614. Atblock 620, if the TSM data point's time is less than the gap's end time, thenrecovery metrics processor 520 processes the TSM data point atblock 622. Atblock 624, the gap's checkpoint is updated. Atblock 626, recovery metrics processor gets the next TSM data point frommetrics queue 502 for the gap. Processing continues withblock 620. - One or more parts of the above implementations may include software and/or a combination of software and hardware. An electronic device stores and transmits (internally and/or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, read only memory (ROM), Flash memory, phase change memory, solid state drives (SSDs)) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals—such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data. For instance, an electronic device may include non-volatile memory (with slower read/write times, e.g., magnetic disks, optical disks, read only memory (ROM), Flash memory, phase change memory, SSDs) and volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)), where the non-volatile memory persists the code/data even when the electronic device is turned off (when power is removed), and the electronic device copies that part of the code that is to be executed by the processor(s) of that electronic device from the non-volatile memory into the volatile memory of that electronic device during operation because volatile memory typically has faster read/write times. As another example, an electronic device may include a non-volatile memory (e.g., phase change memory) to store the code/data when the electronic device is turned off, and that same non-volatile memory has sufficiently fast read/write times such that, rather than copying the part of the code to be executed into volatile memory, the code/data may be provided directly to the processor(s) (e.g., loaded into a cache of the processor(s)); in other words, this non-volatile memory operates as both long term storage and main memory, and thus the electronic device may have no or only a small amount of DRAM for main memory. Typical electronic devices also include a set of one or more physical network interface(s) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices.
-
FIG. 7 illustrates anelectronic device 704 according to some implementations.FIG. 7 includeshardware 740 comprising a set of one or more processor(s) 742, a set or one or more network interfaces 744 (wireless and/or wired), and non-transitory machine-readable storage media 748 having stored thereinsoftware 750.TSM data processor 104 may be implemented in one or moreelectronic devices 704. In one implementation, each one of a plurality of consumer devices is implemented in a separate one of the electronic devices 704 (e.g., in an end user electronic device operated by an end user; in which case, thesoftware 750 in each such end user electronic device includes the software to implement one of the end user clients, including software to interface with a cloud computing service (e.g., an application programming interface (API), a web browser, a native client, a portal, a command-line interface, etc.)) andTSM data processor 104 is implemented in a separate set of one or more of the electronic devices 704 (in which case, thesoftware 750 is the software to implement TSM data point processing as described herein; in operation, the end user electronic devices and the electronic device(s) implementing the cloud computing system containingTSM data processor 104 would be commutatively coupled (e.g., by a network) and would establish between them connections for requesting and providing cloud computing services. Other configurations of electronic devices may be used in other implementations. - In electronic devices that use compute virtualization, the processor(s) 742 typically execute software to instantiate a
virtualization layer 754 and software container(s) 762A-R (e.g., with operating system-level virtualization, thevirtualization layer 754 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation ofmultiple software containers 762A-R (representing separate user space instances and also called virtualization engines, virtual private servers, or jails) that may each be used to execute a set of one or more applications; with full virtualization, thevirtualization layer 754 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system (OS), and thesoftware containers 762A-R each represent a tightly isolated form of software container called a virtual machine that is run by the hypervisor and may include a guest operating system; with para-virtualization, an operating system or application running with a virtual machine may be aware of the presence of virtualization for optimization purposes). Again, in electronic devices where compute virtualization is used, during operation an instance of the software 750 (illustrated asinstance 776A) is executed within thesoftware container 762A on thevirtualization layer 754. In electronic devices where compute virtualization is not used, theinstance 776A on top of a host operating system is executed on the “bare metal”electronic device 704. The instantiation of theinstance 776A, as well as thevirtualization layer 754 andsoftware containers 762A-R if implemented, are collectively referred to as software instance(s) 552. - Alternative implementations of an electronic device may have numerous variations from that described above. For example, customized hardware and/or accelerators might also be used in an electronic device.
- In the above description, numerous specific details such as resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. In other instances, control structures, logic implementations, opcodes, means to specify operands, and full software instruction sequences have not been shown in detail since those of ordinary skill in the art, with the included descriptions, will be able to implement what is described without undue implementation.
- References in the specification to “one implementation,” “an implementation,” “an example implementation,” “some implementations,” etc., indicate that the implementation described may include a particular feature, structure, or characteristic, but every implementation may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an implementation, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other implementations whether or not explicitly described.
- Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dot-dash, and dots) may be used herein to illustrate optional operations and/or structures that add additional features to some implementations. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain implementations.
- In the following description and claims, the term “coupled,” along with its derivatives, may be used. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other.
- While the flow diagrams in the figures show a particular order of operations performed by certain implementations, it should be understood that such order is exemplary (e.g., alternative implementations may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).
- While the above description includes several exemplary implementations, those skilled in the art will recognize that the invention is not limited to the implementations described and can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus illustrative instead of limiting.
Claims (26)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/396,516 US11520759B2 (en) | 2019-04-26 | 2019-04-26 | Processing time series metrics data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/396,516 US11520759B2 (en) | 2019-04-26 | 2019-04-26 | Processing time series metrics data |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200341956A1 true US20200341956A1 (en) | 2020-10-29 |
US11520759B2 US11520759B2 (en) | 2022-12-06 |
Family
ID=72917143
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/396,516 Active 2041-06-09 US11520759B2 (en) | 2019-04-26 | 2019-04-26 | Processing time series metrics data |
Country Status (1)
Country | Link |
---|---|
US (1) | US11520759B2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11388077B2 (en) * | 2019-10-30 | 2022-07-12 | Netspective Communications Llc | Computer-executable and traceable metric queues system |
US20230298404A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US11929878B2 (en) | 2019-09-20 | 2024-03-12 | Sonatus, Inc. | System, method, and apparatus for extra vehicle communications control |
US11947540B1 (en) * | 2022-06-30 | 2024-04-02 | Amazon Technologies, Inc. | Query language for metric data |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160034504A1 (en) * | 2014-07-31 | 2016-02-04 | AppDynamics, Inc. | Efficient aggregation, storage and querying of large volume metrics |
US20160189186A1 (en) * | 2014-12-29 | 2016-06-30 | Google Inc. | Analyzing Semantic Places and Related Data from a Plurality of Location Data Reports |
US20170324802A1 (en) * | 2016-05-04 | 2017-11-09 | Secureworks Corp. | System and Method to Access Aggregated Metric Data in a Computer Network |
US20190146978A1 (en) * | 2017-11-15 | 2019-05-16 | Sumo Logic | Key name synthesis |
US10944655B2 (en) * | 2014-07-31 | 2021-03-09 | Cisco Technology, Inc. | Data verification based upgrades in time series system |
Family Cites Families (95)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5649104A (en) | 1993-03-19 | 1997-07-15 | Ncr Corporation | System for allowing user of any computer to draw image over that generated by the host computer and replicating the drawn image to other computers |
US5608872A (en) | 1993-03-19 | 1997-03-04 | Ncr Corporation | System for allowing all remote computers to perform annotation on an image and replicating the annotated image on the respective displays of other comuters |
US5577188A (en) | 1994-05-31 | 1996-11-19 | Future Labs, Inc. | Method to provide for virtual screen overlay |
GB2300991B (en) | 1995-05-15 | 1997-11-05 | Andrew Macgregor Ritchie | Serving signals to browsing clients |
US5715450A (en) | 1995-09-27 | 1998-02-03 | Siebel Systems, Inc. | Method of selecting and presenting data from a database using a query language to a user of a computer system |
US5831610A (en) | 1996-02-23 | 1998-11-03 | Netsuite Development L.P. | Designing networks |
US5821937A (en) | 1996-02-23 | 1998-10-13 | Netsuite Development, L.P. | Computer method for updating a network design |
US5873096A (en) | 1997-10-08 | 1999-02-16 | Siebel Systems, Inc. | Method of maintaining a network of partially replicated database system |
US6604117B2 (en) | 1996-03-19 | 2003-08-05 | Siebel Systems, Inc. | Method of maintaining a network of partially replicated database system |
WO1998038587A1 (en) | 1997-02-26 | 1998-09-03 | Siebel Systems, Inc. | Method of using a cache to determine the visibility to a remote database client of a plurality of database transactions |
AU6668398A (en) | 1997-02-26 | 1998-09-18 | Siebel Systems, Inc. | Method of determining the visibility to a remote databaseclient of a plurality of database transactions using simplified visibility rules |
AU6654798A (en) | 1997-02-26 | 1998-09-18 | Siebel Systems, Inc. | Method of determining visibility to a remote database client of a plurality of database transactions using a networked proxy server |
WO1998038583A1 (en) | 1997-02-26 | 1998-09-03 | Siebel Systems, Inc. | Method of determining visibility to a remote database client of a plurality of database transactions having variable visibility strengths |
WO1998040804A2 (en) | 1997-02-26 | 1998-09-17 | Siebel Systems, Inc. | Distributed relational database |
AU6336798A (en) | 1997-02-27 | 1998-09-29 | Siebel Systems, Inc. | Method of synchronizing independently distributed software and database schema |
EP1019807B1 (en) | 1997-02-27 | 2017-04-05 | Siebel Systems, Inc. | Method of migrating to a successive level of a software distribution incorporating local modifications |
JP2001513926A (en) | 1997-02-28 | 2001-09-04 | シーベル システムズ,インコーポレイティド | Partially replicated distributed database with multiple levels of remote clients |
US6169534B1 (en) | 1997-06-26 | 2001-01-02 | Upshot.Com | Graphical user interface for customer information management |
US5918159A (en) | 1997-08-04 | 1999-06-29 | Fomukong; Mundi | Location reporting satellite paging system with optional blocking of location reporting |
US6560461B1 (en) | 1997-08-04 | 2003-05-06 | Mundi Fomukong | Authorized location reporting paging system |
US20020059095A1 (en) | 1998-02-26 | 2002-05-16 | Cook Rachael Linette | System and method for generating, capturing, and managing customer lead information over a computer network |
US6732111B2 (en) | 1998-03-03 | 2004-05-04 | Siebel Systems, Inc. | Method, apparatus, system, and program product for attaching files and other objects to a partially replicated database |
US5963953A (en) | 1998-03-30 | 1999-10-05 | Siebel Systems, Inc. | Method, and system for product configuration |
JP2002523842A (en) | 1998-08-27 | 2002-07-30 | アップショット・コーポレーション | Method and apparatus for network-based sales force management |
US6549908B1 (en) | 1998-11-18 | 2003-04-15 | Siebel Systems, Inc. | Methods and apparatus for interpreting user selections in the context of a relation distributed as a set of orthogonalized sub-relations |
US6728960B1 (en) | 1998-11-18 | 2004-04-27 | Siebel Systems, Inc. | Techniques for managing multiple threads in a browser environment |
US6601087B1 (en) | 1998-11-18 | 2003-07-29 | Webex Communications, Inc. | Instant document sharing |
AU1838300A (en) | 1998-11-30 | 2000-06-19 | Siebel Systems, Inc. | Smart scripting call centers |
AU2034500A (en) | 1998-11-30 | 2000-06-19 | Siebel Systems, Inc. | State models for monitoring processes |
JP2002531890A (en) | 1998-11-30 | 2002-09-24 | シーベル システムズ,インコーポレイティド | Development tools, methods and systems for client-server applications |
AU2707200A (en) | 1998-11-30 | 2000-06-19 | Siebel Systems, Inc. | Assignment manager |
US6574635B2 (en) | 1999-03-03 | 2003-06-03 | Siebel Systems, Inc. | Application instantiation based upon attributes and values stored in a meta data repository, including tiering of application layers objects and components |
US20020072951A1 (en) | 1999-03-03 | 2002-06-13 | Michael Lee | Marketing support database management method, system and program product |
US6621834B1 (en) | 1999-11-05 | 2003-09-16 | Raindance Communications, Inc. | System and method for voice transmission over network protocols |
US6535909B1 (en) | 1999-11-18 | 2003-03-18 | Contigo Software, Inc. | System and method for record and playback of collaborative Web browsing session |
US6324568B1 (en) | 1999-11-30 | 2001-11-27 | Siebel Systems, Inc. | Method and system for distributing objects over a network |
US6654032B1 (en) | 1999-12-23 | 2003-11-25 | Webex Communications, Inc. | Instant sharing of documents on a remote server |
US6577726B1 (en) | 2000-03-31 | 2003-06-10 | Siebel Systems, Inc. | Computer telephony integration hotelling method and system |
US6732100B1 (en) | 2000-03-31 | 2004-05-04 | Siebel Systems, Inc. | Database access method and system for user role defined access |
US7266502B2 (en) | 2000-03-31 | 2007-09-04 | Siebel Systems, Inc. | Feature centric release manager method and system |
US6336137B1 (en) | 2000-03-31 | 2002-01-01 | Siebel Systems, Inc. | Web client-server system and method for incompatible page markup and presentation languages |
US7730072B2 (en) | 2000-04-14 | 2010-06-01 | Rightnow Technologies, Inc. | Automated adaptive classification system for knowledge networks |
US6665655B1 (en) | 2000-04-14 | 2003-12-16 | Rightnow Technologies, Inc. | Implicit rating of retrieved information in an information search system |
US6842748B1 (en) | 2000-04-14 | 2005-01-11 | Rightnow Technologies, Inc. | Usage based strength between related information in an information retrieval system |
US6434550B1 (en) | 2000-04-14 | 2002-08-13 | Rightnow Technologies, Inc. | Temporal updates of relevancy rating of retrieved information in an information search system |
US6763501B1 (en) | 2000-06-09 | 2004-07-13 | Webex Communications, Inc. | Remote document serving |
KR100365357B1 (en) | 2000-10-11 | 2002-12-18 | 엘지전자 주식회사 | Method for data communication of mobile terminal |
US7581230B2 (en) | 2001-02-06 | 2009-08-25 | Siebel Systems, Inc. | Adaptive communication application programming interface |
USD454139S1 (en) | 2001-02-20 | 2002-03-05 | Rightnow Technologies | Display screen for a computer |
US7363388B2 (en) | 2001-03-28 | 2008-04-22 | Siebel Systems, Inc. | Method and system for direct server synchronization with a computing device |
US6829655B1 (en) | 2001-03-28 | 2004-12-07 | Siebel Systems, Inc. | Method and system for server synchronization with a computing device via a companion device |
US7174514B2 (en) | 2001-03-28 | 2007-02-06 | Siebel Systems, Inc. | Engine to present a user interface based on a logical structure, such as one for a customer relationship management system, across a web site |
US20030018705A1 (en) | 2001-03-31 | 2003-01-23 | Mingte Chen | Media-independent communication server |
US20030206192A1 (en) | 2001-03-31 | 2003-11-06 | Mingte Chen | Asynchronous message push to web browser |
US6732095B1 (en) | 2001-04-13 | 2004-05-04 | Siebel Systems, Inc. | Method and apparatus for mapping between XML and relational representations |
US7761288B2 (en) | 2001-04-30 | 2010-07-20 | Siebel Systems, Inc. | Polylingual simultaneous shipping of software |
US6763351B1 (en) | 2001-06-18 | 2004-07-13 | Siebel Systems, Inc. | Method, apparatus, and system for attaching search results |
US6711565B1 (en) | 2001-06-18 | 2004-03-23 | Siebel Systems, Inc. | Method, apparatus, and system for previewing search results |
US6728702B1 (en) | 2001-06-18 | 2004-04-27 | Siebel Systems, Inc. | System and method to implement an integrated search center supporting a full-text search and query on a database |
US6782383B2 (en) | 2001-06-18 | 2004-08-24 | Siebel Systems, Inc. | System and method to implement a persistent and dismissible search center frame |
US20030004971A1 (en) | 2001-06-29 | 2003-01-02 | Gong Wen G. | Automatic generation of data models and accompanying user interfaces |
US6826582B1 (en) | 2001-09-28 | 2004-11-30 | Emc Corporation | Method and system for using file systems for content management |
US6978445B2 (en) | 2001-09-28 | 2005-12-20 | Siebel Systems, Inc. | Method and system for supporting user navigation in a browser environment |
US7761535B2 (en) | 2001-09-28 | 2010-07-20 | Siebel Systems, Inc. | Method and system for server synchronization with a computing device |
US6993712B2 (en) | 2001-09-28 | 2006-01-31 | Siebel Systems, Inc. | System and method for facilitating user interaction in a browser environment |
US6724399B1 (en) | 2001-09-28 | 2004-04-20 | Siebel Systems, Inc. | Methods and apparatus for enabling keyboard accelerators in applications implemented via a browser |
US7962565B2 (en) | 2001-09-29 | 2011-06-14 | Siebel Systems, Inc. | Method, apparatus and system for a mobile web client |
US7146617B2 (en) | 2001-09-29 | 2006-12-05 | Siebel Systems, Inc. | Method, apparatus, and system for implementing view caching in a framework to support web-based applications |
US8359335B2 (en) | 2001-09-29 | 2013-01-22 | Siebel Systems, Inc. | Computing system and method to implicitly commit unsaved data for a world wide web application |
US6901595B2 (en) | 2001-09-29 | 2005-05-31 | Siebel Systems, Inc. | Method, apparatus, and system for implementing a framework to support a web-based application |
US7289949B2 (en) | 2001-10-09 | 2007-10-30 | Right Now Technologies, Inc. | Method for routing electronic correspondence based on the level and type of emotion contained therein |
US6804330B1 (en) | 2002-01-04 | 2004-10-12 | Siebel Systems, Inc. | Method and system for accessing CRM data via voice |
US7058890B2 (en) | 2002-02-13 | 2006-06-06 | Siebel Systems, Inc. | Method and system for enabling connectivity to a data system |
US7672853B2 (en) | 2002-03-29 | 2010-03-02 | Siebel Systems, Inc. | User interface for processing requests for approval |
US7131071B2 (en) | 2002-03-29 | 2006-10-31 | Siebel Systems, Inc. | Defining an approval process for requests for approval |
US6850949B2 (en) | 2002-06-03 | 2005-02-01 | Right Now Technologies, Inc. | System and method for generating a dynamic interface via a communications network |
US7594181B2 (en) | 2002-06-27 | 2009-09-22 | Siebel Systems, Inc. | Prototyping graphical user interfaces |
US8639542B2 (en) | 2002-06-27 | 2014-01-28 | Siebel Systems, Inc. | Method and apparatus to facilitate development of a customer-specific business process model |
US7437720B2 (en) | 2002-06-27 | 2008-10-14 | Siebel Systems, Inc. | Efficient high-interactivity user interface for client-server applications |
US20040010489A1 (en) | 2002-07-12 | 2004-01-15 | Rightnow Technologies, Inc. | Method for providing search-specific web pages in a network computing environment |
US7251787B2 (en) | 2002-08-28 | 2007-07-31 | Siebel Systems, Inc. | Method and apparatus for an integrated process modeller |
US9448860B2 (en) | 2003-03-21 | 2016-09-20 | Oracle America, Inc. | Method and architecture for providing data-change alerts to external applications via a push service |
EP1606740A4 (en) | 2003-03-24 | 2007-10-03 | Siebel Systems Inc | Common common object |
WO2004086197A2 (en) | 2003-03-24 | 2004-10-07 | Siebel Systems, Inc. | Custom common object |
US7904340B2 (en) | 2003-03-24 | 2011-03-08 | Siebel Systems, Inc. | Methods and computer-readable medium for defining a product model |
US8762415B2 (en) | 2003-03-25 | 2014-06-24 | Siebel Systems, Inc. | Modeling of order data |
US7685515B2 (en) | 2003-04-04 | 2010-03-23 | Netsuite, Inc. | Facilitating data manipulation in a browser-based user interface of an enterprise business application |
US7620655B2 (en) | 2003-05-07 | 2009-11-17 | Enecto Ab | Method, device and computer program product for identifying visitors of websites |
US7409336B2 (en) | 2003-06-19 | 2008-08-05 | Siebel Systems, Inc. | Method and system for searching data based on identified subset of categories and relevance-scored text representation-category combinations |
US20040260659A1 (en) | 2003-06-23 | 2004-12-23 | Len Chan | Function space reservation system |
US7237227B2 (en) | 2003-06-30 | 2007-06-26 | Siebel Systems, Inc. | Application user interface template with free-form layout |
US7694314B2 (en) | 2003-08-28 | 2010-04-06 | Siebel Systems, Inc. | Universal application network architecture |
US7289976B2 (en) | 2004-12-23 | 2007-10-30 | Microsoft Corporation | Easy-to-use data report specification |
US8954500B2 (en) | 2008-01-04 | 2015-02-10 | Yahoo! Inc. | Identifying and employing social network relationships |
US11182434B2 (en) * | 2017-11-15 | 2021-11-23 | Sumo Logic, Inc. | Cardinality of time series |
-
2019
- 2019-04-26 US US16/396,516 patent/US11520759B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160034504A1 (en) * | 2014-07-31 | 2016-02-04 | AppDynamics, Inc. | Efficient aggregation, storage and querying of large volume metrics |
US10944655B2 (en) * | 2014-07-31 | 2021-03-09 | Cisco Technology, Inc. | Data verification based upgrades in time series system |
US20160189186A1 (en) * | 2014-12-29 | 2016-06-30 | Google Inc. | Analyzing Semantic Places and Related Data from a Plurality of Location Data Reports |
US20170324802A1 (en) * | 2016-05-04 | 2017-11-09 | Secureworks Corp. | System and Method to Access Aggregated Metric Data in a Computer Network |
US20190146978A1 (en) * | 2017-11-15 | 2019-05-16 | Sumo Logic | Key name synthesis |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230298399A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230298404A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230298403A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230298400A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230298402A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230298405A1 (en) * | 2019-09-20 | 2023-09-21 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230316817A1 (en) * | 2019-09-20 | 2023-10-05 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US20230360448A1 (en) * | 2019-09-20 | 2023-11-09 | Sonatus, Inc. | System, method, and apparatus for managing vehicle data collection |
US11929878B2 (en) | 2019-09-20 | 2024-03-12 | Sonatus, Inc. | System, method, and apparatus for extra vehicle communications control |
US11943109B2 (en) | 2019-09-20 | 2024-03-26 | Sonatus, Inc. | System, method, and apparatus for extra vehicle communications control |
US11388077B2 (en) * | 2019-10-30 | 2022-07-12 | Netspective Communications Llc | Computer-executable and traceable metric queues system |
US11888597B2 (en) | 2019-10-30 | 2024-01-30 | Intellectual Frontiers Llc | Computer-executable and traceable metric queues system |
US11947540B1 (en) * | 2022-06-30 | 2024-04-02 | Amazon Technologies, Inc. | Query language for metric data |
Also Published As
Publication number | Publication date |
---|---|
US11520759B2 (en) | 2022-12-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11520759B2 (en) | Processing time series metrics data | |
US11182356B2 (en) | Indexing for evolving large-scale datasets in multi-master hybrid transactional and analytical processing systems | |
AU2014346369B2 (en) | Managed service for acquisition, storage and consumption of large-scale data streams | |
US10083092B2 (en) | Block level backup of virtual machines for file name level based file search and restoration | |
US8954967B2 (en) | Adaptive parallel data processing | |
US9910742B1 (en) | System comprising front-end and back-end storage tiers, data mover modules and associated metadata warehouse | |
US10127243B2 (en) | Fast recovery using self-describing replica files in a distributed storage system | |
US20150134796A1 (en) | Dynamic partitioning techniques for data streams | |
US10585760B2 (en) | File name level based file search and restoration from block level backups of virtual machines | |
US10838934B2 (en) | Modifying archive data without table changes | |
CN111801661A (en) | Transaction operations in a multi-host distributed data management system | |
US10572178B2 (en) | Expiration handling for block level backup of virtual machines | |
US20210303537A1 (en) | Log record identification using aggregated log indexes | |
US20160203102A1 (en) | Efficient remote pointer sharing for enhanced access to key-value stores | |
US9063980B2 (en) | Log consolidation | |
US11119685B2 (en) | System and method for accelerated data access | |
US11442927B1 (en) | Storage performance-based distribution of deduplicated data to nodes within a clustered storage environment | |
US20190377809A1 (en) | Resolving versions in an append-only large-scale data store in distributed data management systems | |
US20220342888A1 (en) | Object tagging | |
US11526501B2 (en) | Materialized views assistant | |
US10606805B2 (en) | Object-level image query and retrieval | |
US11580110B2 (en) | Methods and apparatuses for generating redo records for cloud-based database | |
US20200334108A1 (en) | System and method for searchable backup data | |
US10872073B1 (en) | Lock-free updates to a data retention index | |
US20220358096A1 (en) | Management of consistent indexes without transactions |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: SALESFORCE.COM, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BABOVICH, SERGEI;BAYER, CHRISTIAN;BABENKO, DMITRY;AND OTHERS;SIGNING DATES FROM 20190423 TO 20190426;REEL/FRAME:049584/0394 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |